Skip to content

[Bugfix] remove duplicate tokens streamed in required tool choice streaming #23312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
chaunceyjiang merged 9 commits into from
Sep 16, 2025
Merged

[Bugfix] remove duplicate tokens streamed in required tool choice streaming #23312

Changes from 8 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
e14c18c
remove duplicate appending of text to previous_texts in required tool…
Jason-CKY Aug 21, 2025
39e7ef1
Merge branch 'main' into fix-required-tool-use-streaming-error
Jason-CKY Aug 21, 2025
a1d599e
Merge branch 'main' into fix-required-tool-use-streaming-error
Jason-CKY Aug 21, 2025
e4d3d6c
Merge branch 'main' into fix-required-tool-use-streaming-error
Jason-CKY Aug 24, 2025
d96f275
Merge branch 'main' into fix-required-tool-use-streaming-error
Jason-CKY Aug 26, 2025
0ec6729
Merge branch 'main' into fix-required-tool-use-streaming-error
Jason-CKY Aug 27, 2025
86f8ad6
Merge branch 'main' into fix-required-tool-use-streaming-error
Jason-CKY Aug 29, 2025
edd0bb2
Merge branch 'main' into fix-required-tool-use-streaming-error
Jason-CKY Sep 2, 2025
3ab27d3
Merge branch 'main' into fix-required-tool-use-streaming-error
chaunceyjiang Sep 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions vllm/entrypoints/openai/serving_chat.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -784,9 +784,6 @@ async def chat_completion_stream_generator(
delta_message.tool_calls[0].id is not None):
history_tool_call_cnt += 1

# update the previous values for the next iteration
previous_texts[i] = current_text

# handle streaming deltas for tools with "auto" tool choice
# and reasoning parser
elif tool_choice_auto and self.reasoning_parser:
Expand Down