fix(claude-stream): always emit closing events when upstream sends finish_reason without usage - #5345
Conversation
…nish_reason
## Problem
When converting OpenAI streaming responses to Claude format, if the
OpenAI-compatible upstream sends a chunk with `finish_reason` but no
`usage` field, and never follows up with a separate usage-only chunk,
the Claude stream is silently truncated — `message_delta` and
`message_stop` events are never emitted.
This violates the Claude Messages SSE protocol, which requires every
stream to end with `message_stop`. As a result, Claude clients (e.g.
Claude Code) hang indefinitely waiting for the stream to terminate.
## Affected Upstreams
This is the protocol behavior of any OpenAI-compatible upstream that
omits `usage` when the client doesn't pass `stream_options.include_usage=true`,
which is fully compliant with the OpenAI spec. Confirmed reproducers:
- LiteLLM proxy
- Custom OpenAI-compatible gateways
- Some Azure OpenAI deployments
## Root Cause
In `service/convert.go`, the `doneChunk` branch deferred emitting
closing events when `usage` was missing, expecting a follow-up
usage-only chunk that never arrives:
if oaiUsage == nil {
oaiUsage = info.ClaudeConvertInfo.Usage
// Defer closing until usage is available...
return claudeResponses // ← stream silently truncated
}
## Fix
1. **service/convert.go**: emit `message_delta` + `message_stop`
immediately in the `doneChunk` branch, regardless of whether
`usage` is available. The `message_delta` event includes usage
when available, omits it otherwise — both forms are valid per
Claude protocol spec.
2. **service/convert.go**: export `BuildClaudeUsageFromOpenAIUsage`
and `StopReasonOpenAI2Claude` (as wrappers) so the fallback path
in `relay/channel/openai/helper.go` can reuse the conversion
logic without duplication.
3. **relay/channel/openai/helper.go**: add fallback closing events
in `HandleFinalResponse` for cases where the stream is truncated
without `finish_reason` at all (e.g. connection dropped, network
timeout). Ensures `message_stop` is always emitted.
## Protocol Responsibility
The Anthropic Messages SSE spec requires every stream to terminate
with `message_stop`:
> The end of the stream is indicated by a `message_stop` event.
> https://docs.anthropic.com/en/api/messages-streaming
new-api is a protocol converter (OpenAI ↔ Claude), and its contract
is to accept any *valid* OpenAI input and produce *valid* Claude
output. The OpenAI input here is fully spec-compliant — the fix
belongs in new-api.
## Testing
Verified locally with:
- LiteLLM proxy (gpt-4) → Claude Code: previously hung, now closes correctly
- Direct OpenAI API → Claude Code: still works (no regression)
- Tool calls / streaming with thinking blocks: still works (no regression)
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
WalkthroughThis PR updates Claude stream termination handling by exporting conversion helpers, changing streaming finish handling to emit closing events immediately, and adding a relay fallback that finalizes incomplete Claude responses. ChangesClaude Stream Termination Safety
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@relay/channel/openai/helper.go`:
- Around line 173-200: The fallback path must close any open Claude content
blocks before emitting the terminal events: check
info.ClaudeConvertInfo.LastMessagesType for an active block and emit the
corresponding content_block_stop event (or call the existing stop-block helper
exported from service/convert.go) prior to sending the
message_delta/message_stop sequence via helper.ClaudeData and
dto.ClaudeResponse; reuse or expose the stop-block logic used in
service/convert.go (the logic around stopping open blocks) and invoke it here so
the event sequence remains valid.
In `@service/convert.go`:
- Around line 479-507: The fast-path that handles info.SendResponseCount == 1
currently only emits a message_delta when usage exists and thus can skip
emitting the terminal message_delta+message_stop when there's a finish_reason
but no usage; update that branch to call the same close-event logic used later
(use stopOpenBlocks(), compute stopReason via
stopReasonOpenAI2Claude(info.FinishReason) with fallback "end_turn", then append
a message_delta containing StopReason and optionally Usage via
buildClaudeUsageFromOpenAIUsage(oaiUsage), followed by a message_stop) so both
the first-chunk fast path and the later multi-chunk path produce the same
terminal message_delta and message_stop and set info.ClaudeConvertInfo.Done =
true.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 08f8c2f6-65ab-44ee-a413-6afcf6438f3c
📒 Files selected for processing (2)
relay/channel/openai/helper.goservice/convert.go
Two issues raised by CodeRabbit review on QuantumNous#5345: 1. **fast-path missed terminal events when usage is absent** `service/convert.go` SendResponseCount==1 branch previously only emitted message_delta when usage was available, so a stream that started and finished in a single chunk with finish_reason but no usage would also hang Claude clients. Now emits message_delta (with or without usage) and message_stop in both paths, matching the multi-chunk doneChunk branch. 2. **fallback path didn't close open content blocks** `relay/channel/openai/helper.go` HandleFinalResponse fallback was sending message_delta / message_stop without first closing any open content_block, breaking the "content_block_start … content_block_stop" pairing required by Claude protocol. Added a new exported helper `service.GenerateClaudeStopBlocksForOpenInfo` that reuses the existing block-tracking state to emit the correct content_block_stop events. The fallback now closes open blocks before sending terminal events, keeping the stream sequence valid. Both fixes maintain the same protocol contract: any valid OpenAI stream input produces a fully spec-compliant Claude SSE output. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
|
Thanks @coderabbitai for the review. Pushed b0707ce04 addressing both points:
Compiles cleanly. Verified the same reproducer (LiteLLM → Claude Code with single-chunk finish_reason and no usage) now closes correctly. |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
# Fetch the diff for commit b0707ce04
gh api repos/QuantumNous/new-api/commits/b0707ce04 --jq '.files[] | {filename: .filename, patch: .patch}' 2>/dev/null | head -400Length of output: 3800
The SSE event ordering is now correct: |
|
LiteLLM + new api + claude code 会无法使用，目前完成合并了吗 |
2 participants
Problem
When converting OpenAI streaming responses to Claude format, if the OpenAI-compatible upstream sends a chunk with
finish_reasonbut nousagefield, and never follows up with a separate usage-only chunk, the Claude stream is silently truncated —message_deltaandmessage_stopevents are never emitted.This violates the Claude Messages SSE protocol, which requires every stream to end with
message_stop. As a result, Claude clients (e.g. Claude Code) hang indefinitely waiting for the stream to terminate.Affected Upstreams
This is the protocol behavior of any OpenAI-compatible upstream that omits
usagewhen the client doesn't passstream_options.include_usage=true, which is fully compliant with the OpenAI spec. Confirmed reproducers:Root Cause
In
service/convert.go, thedoneChunkbranch deferred emitting closing events whenusagewas missing, expecting a follow-up usage-only chunk that never arrives:Fix
service/convert.go: emitmessage_delta+message_stopimmediately in thedoneChunkbranch, regardless of whetherusageis available. Themessage_deltaevent includes usage when available, omits it otherwise — both forms are valid per Claude protocol spec.service/convert.go: exportBuildClaudeUsageFromOpenAIUsageandStopReasonOpenAI2Claude(as wrappers) so the fallback path inrelay/channel/openai/helper.gocan reuse the conversion logic without duplication.relay/channel/openai/helper.go: add fallback closing events inHandleFinalResponsefor cases where the stream is truncated withoutfinish_reasonat all (e.g. connection dropped, network timeout). Ensuresmessage_stopis always emitted.Why new-api Should Fix This
The Anthropic Messages SSE spec requires every stream to terminate with
message_stop:new-api is a protocol converter (OpenAI ↔ Claude), and its contract is to accept any valid OpenAI input and produce valid Claude output. The OpenAI input here is fully spec-compliant — the fix belongs in new-api.
Testing
Verified locally with:
Summary by CodeRabbit
Release Notes