Repository navigation
Conversation
|
Rebased onto current main (54020c3) after the Streaming Parser Engine rewrite landed. Worth being explicit about what changed around this PR: #45701 moved qwen3 parsing into the engine, but the bug this PR fixes still reproduces on main. The engine's flush converts with partial=False, and the one-shot answer comes from that conversion, so the unclosed trailing parameter is still dropped there. Measured on current main before this rebase: (The engine never puts the marker into the accumulated raw body, so the trailing capture runs cleanly to end of buffer. The sibling tail-class PRs for minimax_m2/glm47/deepseek_v4 were covered by the rewrite and closed; qwen3's was not.) Verification on the rebased head: regression input and the fully-closed control both convert identically (no double counting), and the engine qwen3 suite passes 54/54 including the new TestUnclosedLastParameter cases ( |
2d6f194 to
08cdd8b
Compare
|
This pull request has merge conflicts that must be resolved before it can be |
08cdd8b to
fe595e4
Compare
…function without it
_qwen3_arg_converter only flushed a parameter value at </parameter> or
the next <parameter=. When the model closes </function> without closing
the last parameter, the final conversion (partial=False) dropped that
trailing value: non-streaming returned arguments without it, and the
TOOL_CALL_END flush emitted "{}" shorter than what partial conversions
had already streamed, clipping it away client-side (vllm-project#57699).
Run the trailing-parameter capture in both modes instead of only in
partial mode; the accumulated raw body never contains the </function>
marker, so the value runs cleanly to the end of the buffer.
Co-authored-by: Kimi Code <kimi-code@moonshot.cn>
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
fe595e4 to
5b3884a
Compare
|
Rebased onto current main (8ee4069). The conflict mergify flagged no longer exists against the current tree: nothing under vllm/parser/ has moved since the flagged base, so the rebase applied cleanly with no manual resolution.
|
Purpose
Fixes #57699.
_qwen3_arg_converterended a parameter value only at</parameter>or the next<parameter=. Models sometimes close</function>without closing the last parameter, and then:partial=False) dropped the trailing value —argumentscame back without it (with several parameters, the call silently lost the last one);TOOL_CALL_ENDproduced{}— shorter than what was already sent, so nothing more was emitted and the client kept the clipped version.Not a duplicate: the two earlier qwen3 parser fixes I checked (#43243's JSON-first parsing, and the
null-handling change) cover different shapes; nothing open touches this one.The fix is to run the trailing-parameter capture in both modes instead of only in partial mode. The accumulated raw body never contains the
</function>marker (the engine consumes it as a terminal event), so the trailing value runs cleanly to the end of the buffer. Empty trailing values behave exactly like explicitly-closed empty parameters.Test Plan
Three new tests pin the shape: the BFCL example from the issue (non-streaming), earlier params kept with an unclosed tail (non-streaming), and no truncation in the streamed flush.
Test Result
{}/drops the tail; the streamed flush truncates), pass with the fix.tests/parser/engine/test_qwen3.py: 54 passed (51 existing + 3 new).tests/parser/: 4608 passed with-p no:randomly(the default random-order run surfaces a pre-existing ordering-sensitive flake intests/parser/mistral/test_reasoning.py, present with and without this change; that file passes standalone in both states).tests/parser/cohereneeds the optionalcohere_melodypackage, not installed here.AI assistance: prepared with AI assistance (Kimi Code). I reviewed every changed line and ran the tests above.