fix: support Claude SSE format for ollama cloudflare xai cohere baidu tencent coze dify channels streaming responses

[longkeyy]

Summary

Fix multiple channels to properly output Claude SSE format when request uses Claude format (/v1/messages).

Affected channels:

• ollama
• cloudflare
• xai
• cohere
• baidu
• tencent
• coze
• dify

Each channel now uses openai.HandleStreamFormat() and openai.HandleFinalResponse() which properly switch output format based on info.RelayFormat.

Problem

When using channels with model alias and sending Claude format requests (/v1/messages), the streaming response was incorrectly using OpenAI SSE format:

data: {"id":"...","object":"chat.completion.chunk",...}

Instead of the expected Claude SSE format:

event: message_start
data: {"type":"message_start",...}

event: content_block_delta
data: {"type":"content_block_delta",...}

Root Cause

The stream handlers in these channels were using helper.ObjectData() or helper.StringData() directly without checking info.RelayFormat, which always outputs OpenAI format.

Solution

Use existing openai.HandleStreamFormat() and openai.HandleFinalResponse() functions which already implement proper format switching based on info.RelayFormat (supports OpenAI, Claude, and Gemini formats).

Commits

Each channel has an independent commit for isolation:

1. fix(ollama): support Claude SSE format for streaming responses
2. fix(cloudflare): support Claude SSE format for streaming responses
3. fix(xai): support Claude SSE format for streaming responses
4. fix(cohere): support Claude SSE format for streaming responses
5. fix(baidu): support Claude SSE format for streaming responses
6. fix(tencent): support Claude SSE format for streaming responses
7. fix(coze): support Claude SSE format for streaming responses
8. fix(dify): support Claude SSE format for streaming responses

Test plan

• Test ollama channel with Claude format request (/v1/messages)
• Test other channels with Claude format request
• Verify response uses correct Claude SSE format with proper event types

#1439
#1618
#1854
#2301
#2378

Summary by CodeRabbit

• Refactor

  • Streaming across multiple relay channels now funnels chunked data through a unified stream-format handler and central finalization path so streamed output is consistently serialized and emitted.

• Bug Fixes

  • Eliminated duplicate end/usage signals and consolidated final payload emission to ensure a single, correct final response is delivered for streamed conversations.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

relay/channel/coze/relay-coze.go (1)

225-225: Bug: fmt.Sprintf missing format verbs.

The format string lacks placeholders, so errorData.Code and errorData.Message are silently discarded and won't appear in the log output.

🔎 Proposed fix

-		common.SysLog(fmt.Sprintf("stream event error: ", errorData.Code, errorData.Message))
+		common.SysLog(fmt.Sprintf("stream event error: code=%d, message=%s", errorData.Code, errorData.Message))

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d594cef and bb73d37.

📒 Files selected for processing (4)

• relay/channel/coze/relay-coze.go
• relay/channel/dify/relay-dify.go
• relay/channel/ollama/stream.go
• relay/channel/tencent/relay-tencent.go

🚧 Files skipped from review as they are similar to previous changes (1)

• relay/channel/ollama/stream.go

🧰 Additional context used

🧠 Learnings (2)

📓 Common learnings

Learnt from: jiajunly
Repo: QuantumNous/new-api PR: 1629
File: relay/channel/openai/relay-openai.go:170-174
Timestamp: 2025-08-21T06:31:11.073Z
Learning: In relay/channel/openai/relay-openai.go, the streaming logic for the AddThinkFirst feature is designed so that only the first chunk of a stream gets the "<think>\n" prefix. The final flush in the streaming handler intentionally uses addThink=false because the last chunk should never receive the prefix, even in single-chunk streams where the prefix would have been applied during normal processing.

📚 Learning: 2025-08-21T06:31:11.073Z

Learnt from: jiajunly
Repo: QuantumNous/new-api PR: 1629
File: relay/channel/openai/relay-openai.go:170-174
Timestamp: 2025-08-21T06:31:11.073Z
Learning: In relay/channel/openai/relay-openai.go, the streaming logic for the AddThinkFirst feature is designed so that only the first chunk of a stream gets the "<think>\n" prefix. The final flush in the streaming handler intentionally uses addThink=false because the last chunk should never receive the prefix, even in single-chunk streams where the prefix would have been applied during normal processing.

Applied to files:

• relay/channel/tencent/relay-tencent.go
• relay/channel/dify/relay-dify.go
• relay/channel/coze/relay-coze.go

🧬 Code graph analysis (3)

relay/channel/tencent/relay-tencent.go (1)

service/usage_helpr.go (1)

• ResponseText2Usage (22-29)

relay/channel/dify/relay-dify.go (1)

service/usage_helpr.go (1)

• ResponseText2Usage (22-29)

relay/channel/coze/relay-coze.go (2)

service/usage_helpr.go (1)

• ResponseText2Usage (22-29)

relay/common/relay_info.go (1)

• RelayInfo (81-128)

🔇 Additional comments (12)

relay/channel/dify/relay-dify.go (4)

18-18: LGTM!

Import of the openai package is necessary for the new streaming format helpers (HandleStreamFormat and HandleFinalResponse).

---

156-160: Good fix for nil pointer dereference.

Constructing a DifyFile struct literal instead of attempting to set fields on an uninitialized (nil) pointer correctly resolves the crash for remote images.

---

219-219: LGTM!

The introduction of lastStreamData and the switch to openai.HandleStreamFormat aligns with the unified streaming pattern across other channels. Marshaling the response and storing it for later finalization is the correct approach.

Also applies to: 245-248

---

255-258: LGTM!

The finalization via HandleFinalResponse correctly passes the accumulated lastStreamData, timestamp, model name, and usage. This ensures Claude/Gemini format requests receive the appropriate SSE events.

relay/channel/tencent/relay-tencent.go (4)

21-21: LGTM!

Import of the openai package is required for the new streaming format helpers.

---

96-98: LGTM!

Initializing responseId via helper.GetResponseID(c) at the start ensures a consistent ID across all stream chunks, addressing the empty responseId issue mentioned in the commit summary.

---

119-130: LGTM!

The streaming logic correctly:

1. Assigns the pre-generated responseId to each response chunk for consistency.
2. Captures the created timestamp.
3. Marshals and routes through HandleStreamFormat for format-aware streaming.

---

137-144: LGTM!

The finalization logic correctly computes usage via ResponseText2Usage and delegates to HandleFinalResponse for proper Claude/Gemini SSE event emission.

relay/channel/coze/relay-coze.go (4)

15-15: LGTM!

Import of the openai package is required for the new streaming format helpers.

---

106-106: LGTM!

The introduction of lastStreamData, updated call sites to pass it by reference, and the finalization via HandleFinalResponse correctly implement the unified streaming pattern for Claude/Gemini format support.

Also applies to: 118-118, 138-138, 149-151

---

155-155: LGTM!

The updated signature correctly accepts info and lastStreamData pointer to enable format-aware streaming and state tracking.

---

172-177: Good addition of error logging for marshal failures.

The error logging for JSON marshal failures improves debuggability, which aligns with the commit message for this PR.

Also applies to: 210-215

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

relay/channel/dify/relay-dify.go (1)

256-257: Empty responseId passed to HandleFinalResponse.

An empty string is passed as responseId. For OpenAI format, this will result in an empty ID in the final usage response. For Claude format, the ID comes from the lastStreamData unmarshalling. Consider using helper.GetResponseID(c) for consistency with other channel implementations.

🔎 Suggested fix

 	// Handle final response based on format
-	openai.HandleFinalResponse(c, info, lastStreamData, "", common.GetTimestamp(), info.UpstreamModelName, "", usage, false)
+	openai.HandleFinalResponse(c, info, lastStreamData, helper.GetResponseID(c), common.GetTimestamp(), info.UpstreamModelName, "", usage, false)

📜 Review details

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bb73d37 and dbee76a.

📒 Files selected for processing (8)

• relay/channel/baidu/relay-baidu.go
• relay/channel/cloudflare/relay_cloudflare.go
• relay/channel/cohere/relay-cohere.go
• relay/channel/coze/relay-coze.go
• relay/channel/dify/relay-dify.go
• relay/channel/ollama/stream.go
• relay/channel/tencent/relay-tencent.go
• relay/channel/xai/text.go

🚧 Files skipped from review as they are similar to previous changes (2)

• relay/channel/xai/text.go
• relay/channel/cloudflare/relay_cloudflare.go

🧰 Additional context used

🧠 Learnings (2)

📓 Common learnings

Learnt from: jiajunly
Repo: QuantumNous/new-api PR: 1629
File: relay/channel/openai/relay-openai.go:170-174
Timestamp: 2025-08-21T06:31:11.073Z
Learning: In relay/channel/openai/relay-openai.go, the streaming logic for the AddThinkFirst feature is designed so that only the first chunk of a stream gets the "<think>\n" prefix. The final flush in the streaming handler intentionally uses addThink=false because the last chunk should never receive the prefix, even in single-chunk streams where the prefix would have been applied during normal processing.

📚 Learning: 2025-08-21T06:31:11.073Z

Learnt from: jiajunly
Repo: QuantumNous/new-api PR: 1629
File: relay/channel/openai/relay-openai.go:170-174
Timestamp: 2025-08-21T06:31:11.073Z
Learning: In relay/channel/openai/relay-openai.go, the streaming logic for the AddThinkFirst feature is designed so that only the first chunk of a stream gets the "<think>\n" prefix. The final flush in the streaming handler intentionally uses addThink=false because the last chunk should never receive the prefix, even in single-chunk streams where the prefix would have been applied during normal processing.

Applied to files:

• relay/channel/tencent/relay-tencent.go
• relay/channel/ollama/stream.go
• relay/channel/cohere/relay-cohere.go
• relay/channel/dify/relay-dify.go
• relay/channel/baidu/relay-baidu.go
• relay/channel/coze/relay-coze.go

🧬 Code graph analysis (5)

relay/channel/tencent/relay-tencent.go (6)

relay/helper/common.go (1)

• GetResponseID (158-161)

common/json.go (1)

• Marshal (21-23)

relay/channel/openai/helper.go (2)

• HandleStreamFormat (22-34)
• HandleFinalResponse (197-254)

common/sys_log.go (1)

• SysLog (11-14)

service/usage_helpr.go (1)

• ResponseText2Usage (22-29)

service/http.go (1)

• CloseResponseBodyGracefully (15-23)

relay/channel/ollama/stream.go (4)

relay/helper/common.go (2)

• GenerateStartEmptyResponse (168-184)
• GenerateStopResponse (186-199)

common/json.go (1)

• Marshal (21-23)

relay/channel/openai/helper.go (2)

• HandleStreamFormat (22-34)
• HandleFinalResponse (197-254)

types/relay_format.go (2)

• RelayFormat (3-3)
• RelayFormatOpenAI (6-6)

relay/channel/cohere/relay-cohere.go (3)

common/json.go (1)

• Marshal (21-23)

relay/channel/openai/helper.go (2)

• HandleStreamFormat (22-34)
• HandleFinalResponse (197-254)

service/usage_helpr.go (1)

• ResponseText2Usage (22-29)

relay/channel/dify/relay-dify.go (4)

common/json.go (1)

• Marshal (21-23)

relay/channel/openai/helper.go (2)

• HandleStreamFormat (22-34)
• HandleFinalResponse (197-254)

service/usage_helpr.go (1)

• ResponseText2Usage (22-29)

common/utils.go (1)

• GetTimestamp (261-263)

relay/channel/baidu/relay-baidu.go (3)

relay/helper/common.go (1)

• SetEventStreamHeaders (41-55)

common/json.go (1)

• Marshal (21-23)

relay/channel/openai/helper.go (2)

• HandleStreamFormat (22-34)
• HandleFinalResponse (197-254)

🔇 Additional comments (21)

relay/channel/cohere/relay-cohere.go (3)

14-14: LGTM: Import added for centralized streaming helpers.

The import of the openai package enables use of HandleStreamFormat and HandleFinalResponse for format-aware streaming.

---

88-88: LGTM: Streaming format handling centralized correctly.

The pattern of marshaling each chunk, storing in lastStreamData, and routing through HandleStreamFormat is consistent with the PR's objective to support Claude/Gemini SSE formats.

Also applies to: 157-160

---

169-170: Empty lastStreamData edge case is valid but behavior is intentional.

If a stream ends without any valid chunks, lastStreamData remains an empty string. When HandleFinalResponse is called, the Claude and Gemini format branches will attempt to unmarshal this empty string, which fails and triggers an error log before returning early. While this prevents an error response from reaching the client, this behavior appears intentional: logging the failure and returning gracefully rather than crashing. If this graceful failure is unacceptable for your requirements, the handlers should validate that lastStreamData is non-empty before attempting unmarshaling, or handle the empty case explicitly.

relay/channel/dify/relay-dify.go (2)

156-160: Good fix: Prevents nil pointer dereference for remote images.

The previous code likely attempted to mutate a nil DifyFile pointer. Creating a new struct literal with all required fields is the correct approach.

---

219-219: LGTM: Streaming format handling follows the established pattern.

The implementation correctly captures each chunk in lastStreamData and routes through HandleStreamFormat.

Also applies to: 245-248

relay/channel/tencent/relay-tencent.go (3)

21-21: LGTM: Import added for centralized streaming helpers.

---

96-98: LGTM: Proper response ID and timestamp handling.

Using helper.GetResponseID(c) ensures a consistent response ID across the stream. The implementation correctly captures each chunk and routes through HandleStreamFormat.

Also applies to: 120-120, 125-130

---

137-144: LGTM: Finalization and cleanup are properly ordered.

The sequence of computing usage, calling HandleFinalResponse, and then closing the response body is correct.

relay/channel/baidu/relay-baidu.go (4)

17-17: LGTM: Import added for centralized streaming helpers.

---

119-123: LGTM: Streaming state initialization.

The variables for tracking stream state are properly initialized. SetEventStreamHeaders is called before processing chunks.

---

136-143: LGTM: Response ID and created timestamp captured from upstream.

The implementation correctly captures the response ID and created timestamp from Baidu's response for use in final response generation.

---

146-151: LGTM: Finalization follows the established pattern.

The call to HandleFinalResponse with the captured metadata is correct.

relay/channel/ollama/stream.go (4)

16-16: LGTM: Import added for centralized streaming helpers.

---

80-87: LGTM: Start response handling updated.

The start response is correctly captured in lastStreamData and routed through HandleStreamFormat.

---

152-155: LGTM: Delta chunks properly routed through format handler.

---

167-179: Good: Format-aware stop event handling prevents duplicate events.

The conditional at lines 172-174 ensures the stop event is only sent via HandleStreamFormat for OpenAI format. For Claude/Gemini formats, the stop/finalization is handled by HandleFinalResponse, avoiding duplicate termination events. This aligns with the commit message mentioning "prevent potential duplicate final events in Ollama for Claude/Gemini formats."

relay/channel/coze/relay-coze.go (5)

15-15: LGTM: Import added for centralized streaming helpers.

---

106-106: LGTM: Streaming state properly threaded through event handler.

The lastStreamData variable is passed as a pointer to handleCozeEvent, allowing updates from within the event handler.

Also applies to: 118-118, 138-138

---

149-152: LGTM: Finalization follows the established pattern.

---

155-155: LGTM: Function signature updated with required parameters.

The updated signature correctly includes info (needed for HandleStreamFormat) and lastStreamData pointer.

---

172-177: Good: Error logging added for JSON marshal failures.

The added error logging for marshal failures improves debuggability, as mentioned in the commit messages.

Also applies to: 210-215