[vMaroon]

Overview

A Qwen3 XML tool call that repeats closed parameters makes _qwen3_arg_converter re-scan every parameter on each streaming conversion, so the API server's cost per token grows with the call until the event loop saturates and liveness fails. This parses each closed parameter once. It is the Qwen3 counterpart of #64535 (GLM) and covers every parser on the Qwen3 converter: qwen3_coder, Nemotron-3, Step-3.5, Step5, Seed-OSS, MiniCPM-V and Granite thinking.

Claims

• Streaming a call with N closed parameters matches the parameter regex N times in total. On main every conversion matches all closed parameters again and the value extender scans them twice more: 2,051,331 matches for N = 401 in the new test.
• Streamed deltas and non-streaming results are unchanged.
• For a call looping on two parameters, the cost per 4-token engine step at 64K output chars drops from 6.4 ms to 92 us, and a replay at 200 tok/s no longer fails liveness (main: killed at 335 s).
• On a real server (Qwen3-Coder-30B-A3B, one H200), that loop to 256K chars keeps the API server 100% busy on main from 148 s, and /health answers no probe within 10 s for the last 150 s of the stream. With this PR the API server is 7% busy and /health p99 is 0.9 ms.
• No new config, flag or API. MiMoParser keeps its own converter and is unchanged.

Validation

Check	Command	main 8084f0624d	This PR
New cost test	pytest tests/parser/engine/test_qwen3.py::TestArgValueExtender::test_closed_params_are_parsed_once	AssertionError: assert 2051331 == 401	passed
Qwen3 parser tests	pytest tests/parser/engine/test_qwen3.py	84 passed, 2 failed (the new tests)	86 passed
Parser suite	pytest tests/parser --ignore=tests/parser/mistral/test_tool_calls.py --ignore=tests/parser/test_harmony.py	6570 passed, 93 errors	6574 passed, 93 errors (same set)
Files that need Hugging Face	pytest tests/tool_parsers/test_qwen3coder_tool_parser.py tests/tool_parsers/test_step3p5_tool_parser.py tests/tool_parsers/test_structural_tag_registry.py tests/parser/test_parse.py tests/parser/test_streaming.py tests/parser/test_include_reasoning.py	427 passed	427 passed
Resumed parse against main (below)	random argument texts grown piece by piece, two calls sharing one cache		0 mismatches in 700,709 growth steps
Streaming equivalence (below)	Qwen3 tokenizer, qwen3_coder + nemotron_v3, parse_delta at chunk sizes 1, 3, 4, 7, 64, plus parse		114 of 114 records identical
Cost per step (below)	parameter loop, through OutputProcessor and chat_completion_stream_generator	1741 us at 16K chars, 6435 us at 64K	57 us at 16K chars, 92 us at 64K
Looping stream replay (below)	200 tok/s, liveness 30 s / 5 s / 3	liveness kill at 335 s	no kill in 1200 s, loop 10% busy at 240K tokens
Server run (below)	Qwen3-Coder-30B-A3B-Instruct on one H200, forced parameter loop to 256K chars	API server 394.8 s CPU (90% busy), 16 /health timeouts of 10 s	26.8 s (7%), /health p99 0.90 ms, no timeout

Unit tests, equivalence and replays ran on CPU in vllm-ci-postmerge-repo:074c83a09a0b872cec92e244d91adf2b3c55e41b-arm64-cpu with the tree on PYTHONPATH. The 93 parser-suite errors are the same set on both: 70 in the three files that need Hugging Face access (the fifth row, run with access), and 23 in tests/parser/mistral/test_reasoning.py, which fails the same on main with the image's mistral-common. ruff, the repo's mypy hook and the CI-wiring hook pass.

CI coverage: tests/parser jobs are disabled since #63609, so CI will run the new tests once that job is back.

Resumed parse against main

Each case builds two calls' raw arguments from fragments: whole and split <parameter=...> openers, </parameter> and < / parameter > closers with prefixes of both, repeated names, newlines, >, =, non-ASCII and value text. Each call grows in random 1-9 char pieces through one shared cache. After every piece, the converter with the cache must return main's _qwen3_arg_converter output for partial=True and partial=False, and the extender must be None exactly when main's is and then stream the same text for every later piece. Seeds 1-3, 4,000 cases each.

Streaming equivalence

Each output is tokenized with the Qwen3 tokenizer and streamed as token-id chunks through ParserManager.get_parser("qwen3_coder", "nemotron_v3"). Every delta's content, reasoning, tool-call name and arguments are recorded, plus one non-streaming parse. The outputs: content, CJK content, reasoning, an open Python, HTML and CJK value, a parameter loop, a call loop, an open JSON array, a long value under a repeated name, and 9 repeated-name cases (a different, identical, extending or prefix value, a repeat that is not last, a third occurrence restoring the first value, another name after the repeat, an unclosed repeat, a second call).

Cost per step and looping stream replay

The stream is <tool_call>\n<function=Edit>\n followed by <parameter=old_string>\ntotal = 0\n</parameter>\n<parameter=new_string>\ntotal = 1\n</parameter>\n repeated, with the Qwen3-0.6B tokenizer. Cost is the endpoint's time per 4-token engine step (chat_completion_stream_generator with qwen3_coder and nemotron_v3, prompt with reasoning closed).

Output chars	1K	4K	16K	64K
main	127 us	642 us	1741 us	6435 us
This PR	56 us	51 us	57 us	92 us

The replay feeds the same stream through the streaming parser on a virtual clock: 200 tokens/s in 4-token steps, one parse_delta per event-loop pass for every token emitted since the previous pass, measured parser time doubled for a slower server CPU, and kubelet liveness (30 s period, 5 s timeout, 3 failures) replayed on those passes.

Stream time	main	This PR
60 s, 12K tokens	loop 48% busy	1%
120 s, 24K tokens	92%	1%
210-330 s	liveness fails 3 of 3 (12-70 s waits), kill at 335 s	2-3%, no probe failure
1200 s, 240K tokens, 818K chars		10%

Server run

vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct --tool-call-parser qwen3_coder --enable-auto-tool-choice from the CUDA CI image of main 8084f0624d, one H200 per server, Model Runner V2, no speculative decoding. The PR server runs the same image with vllm/parser/qwen3.py first on PYTHONPATH. A test logits processor forces the token ids of the cost-per-step stream, closed with </function></tool_call>, at temperature 0, so both servers generate the same output; weights, engine, detokenizer, parser and HTTP server are real. The client streams /v1/chat/completions with an Edit tool, probes /health every 0.5 s with a 10 s timeout and samples the API server process's CPU time.

Run	main: API-server CPU, busy, /health p99, timeouts, stream time	This PR
Forced, 64K chars (18,804 tokens)	41.8 s, 49%, 11.4 ms, 0, 86 s	4.0 s, 5%, 0.90 ms, 0, 86 s
Forced, 256K chars (75,153 tokens)	394.8 s, 90%, 10 s, 16, 440 s	26.8 s, 7%, 0.90 ms, 0, 389 s
Forced, 64K chars, 8 streams at once	185.1 s, 96%, 10 s, 12, 193 s	17.5 s, 13%, 1.8 ms, 0, 132 s
Unforced Write of a Python module	0.86 s, 4%, 0.76 ms, 0, 20 s	0.90 s, 5%, 0.72 ms, 0, 20 s

• On main at 256K chars, the API server is 100% busy from 148 s. /health takes over 1 s from 216 s, and from 287 s until the stream ends at 437 s no probe answers within 10 s. A kubelet liveness probe with a 30 s period, 5 s timeout and 3 failures would restart the container by about 380 s, before the call finishes. With 8 streams at 64K chars, no probe answers from 68 s to the end at 193 s.
• The load also slows the streams on main: 440 s against 389 s at 256K chars, and 193 s against 132 s for the 8 streams.
• Streamed arguments are identical between the two servers in every run, and across the 8 streams. The unforced run's arguments parse as JSON on both.

Details

Root cause.

• ParserEngine._compute_arg_delta converts the whole raw argument text whenever the open-value extender is inactive: between parameters, and while the open parameter repeats an earlier name (the extender needs the open value to be the last JSON value). Qwen3 sets no arg_structural_chars, so every argument delta in those states converts.
• _qwen3_arg_converter runs _PARAM_RE.finditer over every closed parameter, then _PARAM_RE.sub over the whole text to find the open one. After each conversion that extends the streamed JSON, _param_value_extender runs both again.
• A model looping on closed parameters keeps every delta in those states, so the cost per delta is linear in the call and the total is quadratic. Merging deltas under load does not bound it, because each merged pass converts the whole text again.

Change.

• _qwen3_parse_params takes an optional _ClosedParams cache, and Qwen3Parser binds one per instance into its converter and extender after super().__init__, as Glm47MoeParser does since [Bugfix][Parser] Parse closed GLM-4.7 tool arguments once while streaming #64535. A text that starts with the cached text resumes after the last closed parameter. Any other text parses from the start, and the cache moves forward whenever a scan closes a new parameter.
• A Qwen3 value also closes where the next <parameter= begins (the lookahead in _PARAM_RE), and holds only while that opener follows. So the cached text runs through that opener, and the scan resumes at the end of the value. Appending text cannot change a parameter closed within the cached text, so resuming is exact. test_value_closed_by_next_parameter_needs_that_parameter covers a text that shares the prefix but not the opener.
• The extender's repeated-name check uses the closed names from the same parse in place of a second finditer.
• A parser built with a custom arg_converter keeps the uncached path (identity check in __init__), as in [Bugfix][Parser] Parse closed GLM-4.7 tool arguments once while streaming #64535.

Cost. What stays linear in the text per conversion is C-level work: startswith against the cached text, the join of text outside the parameters for the open-parameter search, and the engine's own argument concatenation. That is the 10% busy at 818K chars in the replay.

Not in this PR. #64535 also set arg_structural_chars=frozenset(">") for GLM, on the argument that a delta without > cannot close a key or value. That does not hold for Qwen3, where a value closes at <parameter= before any >, and the cache alone passes the replay. Measured with the gate added: 39 us per step at 64K chars for the parameter loop, and 385 us against 1247 us for many distinct parameters.

Unchanged here and the same on main.

• Many distinct parameters in one call: each conversion still serializes every parameter, so the cost per step grows with their number (1.2 ms at 64K chars, against 6.2 ms on main).
• A long value under a repeated name: _param_value_extender refuses any repeated name, so each delta converts the whole value (11.7 ms per step at 64K chars, against 17.6 ms on main). Letting it accept a repeated last name, as GLM's extender does, is a separate change.
• MiMoParser keeps its own converter with the full scan.

Not a duplicate. #55497 (open) narrows the parameter-name group of _PARAM_RE and _PARTIAL_PARAM_RE; the cache works with either pattern and the overlap is textual. #59821 (open) edits Qwen3Parser.__init__ for tool markup in reasoning; whichever lands second rebases. #63277 (open) changes zero-argument calls in the tool adapter. No open PR covers re-parsing closed Qwen3 parameters.

AI assistance. Claude Code was used for the analysis, the change, the tests and this description.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[smiyamoto3]

LGTM. Checked that it mirrors the already merged #64535 for GLM, and the output looks fine.

[sfeng33]

Thanks!

[sfeng33]

/ci run

[github-actions]

✅ Triggered Buildkite CI #98828 for commit 0d25d9d5bda0.