Skip to content

fix(streaming): scrub leaked partial tool-call opener fragments per-delta - #57035

Closed
davidgut1982 wants to merge 1 commit into
NousResearch:mainfrom
davidgut1982:fix/strip-partial-toolcall-fragments
Closed

davidgut1982 wants to merge 1 commit into
NousResearch:mainfrom
davidgut1982:fix/strip-partial-toolcall-fragments

Conversation

@davidgut1982

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #14251 (strip standalone tool-call XML from visible text), which
handled the post-streaming final-text path but explicitly deferred the live
per-delta scrub
. This adds that per-delta scrub so partial tool-call opener
fragments never reach user-facing streamed output.

The bug / how it manifests

Qwen3-class models sometimes begin emitting an XML tool-call opener as ordinary
content (<tool_call>) and then switch to native tool_calls mid-token,
leaving a stray content delta such as "ool_call>" that streams straight to
user-facing output. These tails are never valid prose.

The fix

  • utils.py: add strip_partial_toolcall_fragments(), using a single compiled
    regex for both search and substitution (no search/sub asymmetry) that matches
    the leaked tails of <tool_call> / <function_call> openers.
  • agent/chat_completion_helpers.py: wire it into the stream_delta_callback
    branch of interruptible_streaming_api_call. That branch bypasses
    _fire_stream_delta and therefore did not previously get the scrub. The
    callback and text-record now guard on the scrubbed result, so a pure-fragment
    delta fires no callback and records no text.

Notes

🤖 Generated with Claude Code

…elta

Follow-up to NousResearch#14251, which stripped standalone tool-call XML from the
post-streaming final text but explicitly deferred the live per-delta path.

Qwen3-class models sometimes begin emitting an XML tool-call opener as ordinary
content (`<tool_call>`) and then switch to native `tool_calls` mid-token,
leaving a stray content delta such as `"ool_call>"` that streams straight to
user-facing output.

Add utils.strip_partial_toolcall_fragments(), a single compiled regex used for
both search and substitution (no search/sub asymmetry), matching the leaked
tails of `<tool_call>`/`<function_call>` openers. Wire it into the
stream_delta_callback branch of interruptible_streaming_api_call, which bypasses
_fire_stream_delta and so did not previously get the scrub; guard on the
scrubbed result so a pure-fragment delta fires no callback and records no text.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Jul 2, 2026
@alt-glitch

Copy link
Copy Markdown

This was generated by AI during triage.

Duplicate of #33355 — same new strip_partial_toolcall_fragments() helper in utils.py and the same wiring into the stream_delta_callback branch of chat_completion_helpers.py, both as the deferred per-delta follow-up to #14251. #33355 is the earlier open PR and is broader: it also scrubs the primary run_agent.py::_fire_stream_delta (Path A) content path, whereas this PR only covers the chat_completion_helpers.py branch (Path B). Related: #46330 (Harmony streaming scrubber, adjacent family).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants