[Perf]: Speedup VoxCPM2 TTS performance and Support PagedAttention

[Sy0307]

Summary

Performance optimization for VoxCPM2 TTS native AR pipeline, built on #2658.

Achieves 4.2x speedup (RTF 0.946 → 0.224) on a single H20 GPU with no audio quality loss.

Key changes:

• PagedAttention for base_lm (28 layers) and residual_lm (8 layers), replacing native StaticKVCache
• Selective torch.compile on MLP + o_proj per layer; RMSNorm/RoPE kept eager for precision
• CFM pre-allocated buffers, fused QKV projection, batch-level residual_model forward
• Online serving support in serving_speech.py

Architecture (per AR step):

MiniCPM4PagedForVoxCPM2 (scaffold, 28 layers, PagedAttention + fp32 RoPE)
→ FSQ → MiniCPM4PagedResidualLM (8 layers, PagedAttention, no RoPE)
→ LocDiT (CFM 10-step) → feat_encoder → AudioVAE → 48kHz waveform

Performance (single H20, enforce_eager=true):

Metric	#2658 baseline	This PR
RTF (best)	0.946	0.224

Test Plan

• E2E regression: 3 EN + 5 ZH sentences, Whisper large-v3 ASR 8/8 correct
• RTF stable at 0.22-0.27 after warmup
• 4 concurrent requests: all produce valid audio

[hsliuustc0106]

A few blocking issues before this can move forward:

DCO check failing — please fix your commit sign-offs.

No online serving support. The PR only has offline inference and no online serving path. Per contribution guidelines, new models must support both modes.

Title mismatch. Prefix is [Perf] but this is a full new model integration (1901 LOC, 12 files). Should be [Model].

External runtime dependency. The model requires pip install voxcpm or a source checkout. What is the long-term plan? Any breaking change in the voxcpm package breaks vllm-omni silently.

Scaffold weight cleanup correctness. _free_scaffold_weights() replaces params with empty tensors after first prefill. If a second request triggers a new prefill, the scaffold forward returns zeros — the engine still tries to do KV allocation. The _prefill_completed flag is reset in preprocess() for new requests, but after scaffold weights are freed, the MiniCPM forward will crash or produce garbage for new prefills.

_sliding_vae_decode is dead code. It just calls _full_vae_decode(). Either implement or remove it along with the _ENABLE_SLIDING_VAE flag.

12 env vars for tuning is excessive. Consider consolidating into a config object or at least documenting them centrally.

[hsliuustc0106]

This is a full model addition (12 files, ~1900 LOC) rather than just a perf optimization. The [Perf] prefix is misleading — it adds VoxCPM2 as a new model with examples, tests, stage configs, and model registration.

Key issues:

1. Wrong prefix — should be [Model] not [Perf]. The perf optimizations (CFM pre-allocation, torch.compile, scaffold skip) are part of the model implementation, not a standalone perf PR.
2. No online serving config — only offline example provided. Is there an OpenAI-compatible serving endpoint?
3. DCO check — please verify all commits are signed off.
4. 12 environment variables in voxcpm2_import_utils.py — consider consolidating or documenting why each is needed. That many env vars is a operational burden for deployment.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[Sy0307]

@hsliuustc0106 ' review sugguestions have all solved.

PTAK @linyueqian .

[linyueqian]

Tested on H20 (single GPU, enforce_eager=true, vllm 0.19.0)

Single request + streaming: looks good. RTF ~0.21 on warm runs, audio quality is correct, streaming produces 33 incremental chunks with monotonically growing audio. torch.compile targets all apply cleanly.

Concurrent batching has two bugs.

---

Bug 1: Stop logic fails for batched requests

With 2 concurrent requests, both produce ~58s audio for sentences that should be ~4s. The stop head never triggers, so requests run to _max_decode_steps.

Repro:

engine.generate([
    {"prompt": "The quick brown fox jumps over the lazy dog."},
    {"prompt": "VoxCPM2 uses PagedAttention for language model inference."},
])
# Each request outputs ~58s audio instead of ~4s
# Sequential runs produce correct ~4s audio

I think the issue is in compute_logits() -- _results_queue maps (req_id, stop_logits) pairs by list index to batch position, but the queue ordering may not match the scheduler's token ordering when multiple requests are in flight. The stop signal never reaches the right request.

---

Bug 2: 4-batch prefill crashes with shape mismatch

RuntimeError: The size of tensor a (11) must match the size of tensor b (18)
  at voxcpm2_talker.py:499
  enc_outputs = tts.fsq_layer(enc_out) * feat_mask.unsqueeze(-1) + ...

feat_mask is computed in preprocess() via _build_prefill_inputs() (which may expand the sequence with prompt cache tokens), but req_hidden is sliced from scaffold_hidden using the vllm-visible token count. These lengths diverge when 4 requests are batched.

---

Test matrix:

Feature	Result
Single request (warm)	RTF 0.213
Streaming	33 incremental audio chunks, monotonically growing
2 concurrent	Runs but stop logic broken (58s audio for 4s sentences)
4 concurrent	Crash in _prepare_residual_prefill

Single-request perf is solid. Suggest fixing concurrent batching before merge, or setting max_batch_size: 1 in the stage config and deferring batching to a follow-up PR.

[Sy0307]

Tested on H20 (single GPU, enforce_eager=true, vllm 0.19.0)

Single request + streaming: looks good. RTF ~0.21 on warm runs, audio quality is correct, streaming produces 33 incremental chunks with monotonically growing audio. torch.compile targets all apply cleanly.

Concurrent batching has two bugs.

Both bugs have been fixed. Thanks.

[linyueqian]

Re-tested on H20 (latest commit 2ba6714)

Both batching bugs from my previous review still reproduce. I traced the root cause:

All requests get req_id="default" because gpu_model_runner.py never passes request_id to model.preprocess(). This means every concurrent request shares the same _RequestState -- the second preprocess() call overwrites the first's prefill_masks, and stop signals get mixed across requests.

I pushed a one-line fix in #2746 (req_infos["request_id"] = req_id before the preprocess call). With that fix:

Test	Before #2746	After #2746
Single request	PASS (RTF ~0.21)	PASS
2 concurrent	FAIL (57s + 58s audio)	PASS (2.72s + 5.28s)
4 concurrent	FAIL (shape crash)	FAIL (msgspec crash -- different bug)

The 4-concurrent failure with #2746 applied is a separate orchestrator-level bug: msgspec.ValidationError: cannot unpack non-iterable NoneType object when requests finish at different times and mm_payload contains None audio entries. This isn't VoxCPM2-specific.

Suggestion: Once #2746 is merged, rebase this PR on top of it. Consider setting max_batch_size: 2 in the stage config until the orchestrator msgspec bug is resolved separately.

[linyueqian]

Re-tested with latest commits (727a236) + #2746 on H20

All tests pass with gpu_memory_utilization: 0.9:

Test	Result
Single request	PASS (RTF ~2.0)
2 concurrent	PASS (2.72s + 5.28s)
4 concurrent	PASS (2.72s + 3.36s + 3.36s + 4.00s)

The msgspec fix in 3b45efb works. The remaining blocker is that this PR depends on #2746 (request_id passthrough in gpu_model_runner.py). Without it, all concurrent requests share a single "default" state and batching is completely broken.

Action items before merge:

1. Merge #2746 first (or include the one-line fix in this PR)
2. Change gpu_memory_utilization from 0.3 to 0.9 in voxcpm2.yaml -- at 0.3, 4 concurrent requests OOM on H20 (140GB)

[linyueqian]

LGTM