fix(responses): fix streaming EOF, missing [DONE], reasoning_tokens and created_at float

[TokensZhuanfa]

Problem

When using Codex client (wire_api = "responses") with NewAPI gateway, streaming responses fail with:

ERROR: stream disconnected before completion: failed to parse ResponseCompleted: missing field `reasoning_tokens`
ERROR: Reconnecting... 1/5
json: cannot unmarshal number 1785990796.0 into Go struct field OpenAIResponsesResponse.response.created_at of type int

This causes repeated retries and duplicate responses.

Root Causes

Bug 1: Missing [DONE] terminator in stream

OaiResponsesStreamHandler and OaiChatToResponsesStreamHandler did not call helper.Done(c) at the end of the stream. Clients (e.g. Codex) receive EOF and treat it as an abnormal disconnection, triggering reconnection loops.

Bug 2: Missing reasoning_tokens in output_tokens_details

Some upstreams (e.g. Moonshot/Kimi) report reasoning_tokens at the top level of the usage object:

{"usage":{"prompt_tokens":86,"completion_tokens":417,"total_tokens":503,"reasoning_tokens":398}}

The Usage struct had no reasoning_tokens top-level field and no output_tokens_details field, so the value was silently dropped. Clients parsing response.completed events fail with missing field reasoning_tokens.

Bug 3: created_at float transparent passthrough

In passthrough mode, created_at was forwarded as raw JSON from upstream. Some upstreams serialize it as a float (e.g. 1785990796.0), which Go clients with strict int types cannot parse.

Fix

1. relaykit/dto/openai_response.go

Added OutputTokensDetails *OutputTokenDetails and ReasoningTokens int fields to the Usage struct.

2. relaykit/relayconvert/internal/oai_chat/to_oai_responses_resp.go

UsageFromChatUsage now extracts reasoning_tokens from both top-level and completion_tokens_details, and populates OutputTokensDetails.

3. relay/channel/openai/relay_responses.go

• Non-stream: Re-serialize response body (guarantees integer created_at), populate output_tokens_details
• Stream: Re-serialize each event, extract reasoning_tokens into output_tokens_details, call helper.Done(c) at stream end

4. relay/channel/openai/responses_via_chat.go

Call helper.Done(c) at stream end.

Impact

• /v1/chat/completions — unchanged, no impact
• /v1/responses — all three bugs fixed
• No breaking changes to existing API

Summary by CodeRabbit

• Bug Fixes
  • Improved Responses API usage reporting by consistently capturing reasoning token counts.
  • Added missing output token details to non-streaming and streaming responses.
  • Improved response delivery reliability with safe fallback handling when re-serialization fails.
  • Ensured streaming requests complete cleanly after all events are delivered.

[coderabbitai]

Walkthrough

The change preserves reasoning-token usage in OpenAI Responses payloads. It adds usage fields, converts reasoning-token data, re-serializes non-streaming and streaming responses, falls back to raw data on serialization errors, and signals stream completion.

Changes

Reasoning usage handling

Layer / File(s)	Summary
Usage contract and conversion relaykit/dto/openai_response.go, relaykit/relayconvert/internal/oai_chat/to_oai_responses_resp.go	Usage now supports output_tokens_details and top-level reasoning_tokens. Conversion extracts reasoning tokens from both supported usage locations and populates output-token details.
Response delivery normalization relay/channel/openai/relay_responses.go, relay/channel/openai/responses_via_chat.go	Responses handlers normalize and re-serialize usage data, use raw payloads when serialization fails, add missing output-token details to completion events, and signal stream completion.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• QuantumNous/new-api#6593: Related reasoning-token handling in OpenAI response conversion.

Suggested reviewers: calcium-ion

Poem

I’m a rabbit with tokens to track,
Reasoning details now come back.
Streams close clean, payloads stay bright,
Raw data returns when bytes go wrong—
Hop, hop, the Responses flow is right!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main changes: streaming EOF and [DONE] handling, reasoning token preservation, and created_at normalization.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@relay/channel/openai/relay_responses.go`:
- Around line 115-123: Update the stream response handling around streamResponse
and sendResponsesStreamData so usage normalization, including
output_tokens_details derived from top-level reasoning_tokens, completes before
serialization and sending. Move the marshal/fallback send block after the
normalization logic, preserving the existing raw-data fallback when marshaling
fails.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a1f040b3-0b26-4a18-b1a1-33107757ac5f

📥 Commits

Reviewing files that changed from the base of the PR and between 0ab0202 and c0c7f35.

📒 Files selected for processing (4)

• relay/channel/openai/relay_responses.go
• relay/channel/openai/responses_via_chat.go
• relaykit/dto/openai_response.go
• relaykit/relayconvert/internal/oai_chat/to_oai_responses_resp.go

[coderabbitai]

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Serialize the completion event after usage normalization.

Lines 117-123 send the event before Lines 141-154 add missing output_tokens_details. When an upstream reports only top-level reasoning_tokens, the client receives the original completion event without output_tokens_details.

Process the event usage first. Then marshal and send the event.

Also applies to: 141-155

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@relay/channel/openai/relay_responses.go` around lines 115 - 123, Update the
stream response handling around streamResponse and sendResponsesStreamData so
usage normalization, including output_tokens_details derived from top-level
reasoning_tokens, completes before serialization and sending. Move the
marshal/fallback send block after the normalization logic, preserving the
existing raw-data fallback when marshaling fails.