[Perf][Rust Frontend] Coalesce decoded chunks per engine update

[BugenZhao]

Purpose

Streaming engine updates can carry multiple generated tokens with MTP or speculative decoding. The Rust frontend currently extracts and appends a decoded chunk after every token, then yields one TextDelta for the whole engine update.

This change extracts at most one decoded chunk after each nonterminal engine update. Per-token decoding and stop-string checks remain in place, while terminal updates continue through the existing flush() path.

Local release microbenchmark results for the full incremental decode loop (push_token + decode + next_chunk + append), using the Qwen3.5-0.8B tokenizer, 1,000 requests with 256 output tokens each, CPU 69 affinity, and the median of 7 samples:

MTP batch	CPU time	Loop throughput	Allocations
1	Flat	Flat	Flat
2	-10.7%	+12.0%	-14.2%
4	-16.9%	+20.4%	-18.3%
8	-19.9%	+24.9%	-18.2%
16	-20.9%	+26.4%	-18.3%

The benchmark harness was temporary and is excluded from this PR. End-to-end gains depend on the share of frontend CPU spent in incremental detokenization.

Test Plan

• cargo nextest run -p vllm-text
• cargo clippy -p vllm-text --all-targets -- -D warnings
• git diff --check origin/main...HEAD

Test Result

• vllm-text: 90 passed, 1 skipped.
• Clippy passed with warnings denied.
• Diff check passed.
• Model evaluation is not applicable because decoded text, token IDs, logprobs, stop handling, and streaming event boundaries remain unchanged.

AI assistance was used for code changes, validation, benchmarking, and PR preparation. Every changed line and the reported benchmark scope were reviewed by the submitter.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: ed43fed4-0f59-453c-a7d3-4841b1769a76

📥 Commits

Reviewing files that changed from the base of the PR and between a56654d and 6ac4e6b.

📒 Files selected for processing (1)

• rust/src/text/src/output/decoded.rs

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

---

📝 Walkthrough

Walkthrough

The decoded text event stream now skips intermediate chunk appending when a finish reason exists. The terminal flush path emits the final decoded text.

Changes

Decoded output handling

Layer / File(s)	Summary
Guard terminal chunk coalescing rust/src/text/src/output/decoded.rs	The intermediate coalescing branch now appends decoder.next_chunk() output only when finish_reason is None. Terminal updates use the flush path to append the final chunk.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 6ac4e

The change coalesces decoded chunks per engine update while preserving terminal flushing and existing streaming behavior. No actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: coalescing decoded chunks per engine update in the Rust frontend.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[BugenZhao]

/ci run

[github-actions]

✅ Triggered Buildkite CI #86935 for commit 6ac4e6b36cb3.