Skip to content

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented Sep 16, 2025

The streaming response from OpenAIChatCompletionClient.create_stream() was generating many spurious <think> and </think> tags when models returned empty string reasoning content chunks.

Problem

When using models like Qwen3-30B-A3B-Thinking that emit reasoning content during streaming, the API sometimes sends chunks with empty string reasoning_content values (""). The original logic incorrectly treated these empty strings as a signal to exit reasoning mode:

elif is_reasoning:
    # Exit reasoning mode - BUG: triggers for empty strings!
    reasoning_content = "</think>"
    # ...

This caused the streaming output to contain many unwanted tag pairs like:

<think>First part</think><think>Second part</think><think>Third part</think>

Instead of the expected:

<think>First partSecond partThird part</think>

Solution

Modified the exit condition to only trigger when reasoning_content is explicitly None:

elif reasoning_content is None and is_reasoning:
    # Exit reasoning mode only when reasoning_content is None (not when it's an empty string).
    reasoning_content = "</think>"
    # ...

Now empty string reasoning content is ignored (no action taken), and reasoning mode only exits when the API sends None as the proper termination signal.

Testing

Validated the fix with comprehensive tests showing:

  • Before: Multiple spurious </think> tags generated for each empty string chunk
  • After: Exactly one </think> tag generated only when reasoning actually ends

This minimal change preserves all existing functionality while fixing the reported streaming issue.

Fixes #7009.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@Copilot Copilot AI changed the title [WIP] when use OpenAIChatCompletionClient.create_stream(), there is many <think> and </think> tags in content Fix spurious </think> tags caused by empty string reasoning_content in streaming Sep 16, 2025
@Copilot Copilot AI requested a review from ekzhu September 16, 2025 08:19
Copilot finished work on behalf of ekzhu September 16, 2025 08:19
@ekzhu ekzhu marked this pull request as ready for review September 16, 2025 08:38
@ekzhu
Copy link
Collaborator

ekzhu commented Sep 16, 2025

@coach00 pls let us know if this fix works for you.

Copy link

codecov bot commented Sep 16, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.93%. Comparing base (5216bf3) to head (ea9fae1).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #7025   +/-   ##
=======================================
  Coverage   80.93%   80.93%           
=======================================
  Files         237      237           
  Lines       18240    18240           
=======================================
  Hits        14762    14762           
  Misses       3478     3478           
Flag Coverage Δ
unittests 80.93% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ekzhu ekzhu merged commit 79d5d6a into main Sep 16, 2025
76 checks passed
@ekzhu ekzhu deleted the copilot/fix-7009 branch September 16, 2025 09:06
@coach00
Copy link

coach00 commented Sep 17, 2025

@ekzhu i have tried this fix and it works ! thanks very much !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

when use OpenAIChatCompletionClient.create_stream(), there is many <think> and </think> tags in content
3 participants