[xiaohaifeng1999]

Overview

When a tool-call message arrives in one streamed delta, ernie45 emits only the first call, while jamba and granite-20b-fc emit none or mangle the calls. This change makes each parser emit every call completed by a delta. Fixes #48294.

Claims

• All three parsers emit every tool call completed in a single delta (async scheduling, stream_interval > 1) instead of dropping or mangling calls.
• Coarse delta splits no longer lose calls, emit name-less argument fragments, or strand arguments.
• ernie45 streaming no longer leaks text between tool calls into content; leading/trailing newlines are normalized in the tests.
• jamba and granite-20b-fc skip empty argument diffs; all three return None instead of empty DeltaMessages.

Validation

Measured on 0db3734d51, the branch point of this change. Upstream main has since advanced 64 commits; none touch these six files.

Through the in-tree streaming harness (run_tool_extraction_streaming):

• ernie45, one delta with 2 complete blocks: 1/2 calls. 3 blocks: 1/3. 2-delta split: 1/2.
• jamba, one delta with a 2-element array: 0/2. 2-delta split: 0/2. 3-delta split: 1/2, and the survivor carries empty arguments.
• granite-20b-fc, one delta: 0/2. 2-delta split: 1/2 with a name-less argument fragment. 3-delta split: 2/2, but one call has no name and the other has no arguments.

New regression tests: 19 cases, 16 fail on that base and all pass with this change. The other 3 guard the new semantics (no empty argument deltas, no empty trailing DeltaMessage) and pass before and after.

python -m pytest tests/tool_parsers/test_ernie45_moe_tool_parser.py \
  tests/tool_parsers/test_jamba_tool_parser.py \
  tests/tool_parsers/test_granite_20b_fc_tool_parser.py -q
52 passed, 1 xfailed

The xfail is a pre-existing granite surrounding-text test, unchanged by this PR. Locally I also set HF_ENDPOINT=https://hf-mirror.com, since huggingface.co is unreachable from my network.

No model-quality eval: the model's generated tokens and the non-streaming extraction are unchanged. This change only affects how already-complete calls are chunked in streaming, which the suites above cover.

Details

Root cause, per parser:

• ernie45 drained at most one complete block per call and returned immediately; the rest stayed in the internal buffer.
• jamba's "starting a new tool" branch updated its cursor and returned None without emitting anything.
• granite-20b-fc jumped current_tool_id to the last index, skipping whole calls, and its flush emitted argument fragments without a name.

Fix:

• ernie45 drains every complete block found in the buffer (revives the loop from [Bugfix][Tool Parser] Stream tool calls that arrive whole in a single delta in llama3_json, jamba and ernie45 parsers #48295).
• jamba and granite-20b-fc: the "starting a new tool" branch emits every newly completed call in full, and the name branch co-sends complete arguments when the call is already complete. Slots and prev-state stay in sync.
• jamba and granite-20b-fc skip empty argument diffs; all three return None for deltas with nothing to send.

Prior work: #48295 is the only open PR referencing #48294. It has been inactive since July (stale, needs rebase, no author response); we are posting a heads-up comment there as this PR opens.

• ernie45: [Bugfix][Tool Parser] Stream tool calls that arrive whole in a single delta in llama3_json, jamba and ernie45 parsers #48295 (by @abhay-codes07, validated by @ErenAta16 against July main) proposed the same drain loop; this change revives it with credit.
• jamba: [Bugfix][Tool Parser] Stream tool calls that arrive whole in a single delta in llama3_json, jamba and ernie45 parsers #48295 guarded only the first-call shape, so coarse mid-stream deltas still lose calls; this change fixes the "starting a new tool" branch itself.
• llama3_json from [Bugfix][Tool Parser] Stream tool calls that arrive whole in a single delta in llama3_json, jamba and ernie45 parsers #48295 no longer applies: the legacy parser was removed in the engine migration.
• granite-20b-fc: [Bugfix][Tool Parser] Stream full single-delta tool calls #47955 was never merged; @smiyamoto3's "drop the granite part" remark there referred to the engine-migrated granite_tool_parser.py, not granite-20b-fc. Its guard handled only the first-call shape; this change also handles mid-stream coarse deltas.

gigachat3 shows the same family of bug but is out of scope here; #51702 covers that parser.

Tradeoff: a coarse delta now produces one chunk carrying several complete calls. That matches the non-streaming path and the shape both prior PRs proposed. Fine-grained streaming is unchanged; the existing tests all pass.

The fix and tests were written with AI assistance (GLM). I reviewed every changed line across two review rounds and verified the runs and artifacts above.

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Per-parser fixes stay in the parsers' own files per vllm/tool_parsers/AGENTS.md; no engine or interface changes.

• Testing and Validation: 19 new streaming tests; 16 fail on the base commit, all pass with the change; suites run against real tokenizers where available.

• Code Quality and Style: ruff check and format clean; comments limited to non-obvious decisions.

• Pull Request Contents: Root cause, evidence, prior-work credit, scope, and tradeoffs included above.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀