fix(streaming): scrub leaked partial tool-call opener fragments per-delta - #57035
Closed
davidgut1982 wants to merge 1 commit into
Closed
davidgut1982 wants to merge 1 commit into
davidgut1982 wants to merge 1 commit into
Conversation
…elta Follow-up to NousResearch#14251, which stripped standalone tool-call XML from the post-streaming final text but explicitly deferred the live per-delta path. Qwen3-class models sometimes begin emitting an XML tool-call opener as ordinary content (`<tool_call>`) and then switch to native `tool_calls` mid-token, leaving a stray content delta such as `"ool_call>"` that streams straight to user-facing output. Add utils.strip_partial_toolcall_fragments(), a single compiled regex used for both search and substitution (no search/sub asymmetry), matching the leaked tails of `<tool_call>`/`<function_call>` openers. Wire it into the stream_delta_callback branch of interruptible_streaming_api_call, which bypasses _fire_stream_delta and so did not previously get the scrub; guard on the scrubbed result so a pure-fragment delta fires no callback and records no text. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Duplicate of #33355 — same new |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #14251 (strip standalone tool-call XML from visible text), which
handled the post-streaming final-text path but explicitly deferred the live
per-delta scrub. This adds that per-delta scrub so partial tool-call opener
fragments never reach user-facing streamed output.
The bug / how it manifests
Qwen3-class models sometimes begin emitting an XML tool-call opener as ordinary
content (
<tool_call>) and then switch to nativetool_callsmid-token,leaving a stray content delta such as
"ool_call>"that streams straight touser-facing output. These tails are never valid prose.
The fix
utils.py: addstrip_partial_toolcall_fragments(), using a single compiledregex for both search and substitution (no search/sub asymmetry) that matches
the leaked tails of
<tool_call>/<function_call>openers.agent/chat_completion_helpers.py: wire it into thestream_delta_callbackbranch of
interruptible_streaming_api_call. That branch bypasses_fire_stream_deltaand therefore did not previously get the scrub. Thecallback and text-record now guard on the scrubbed result, so a pure-fragment
delta fires no callback and records no text.
Notes
are removed.
🤖 Generated with Claude Code