[Feature][TTS] Port static chunk ramp to Moss-TTS streaming codec decode

[Wallbreazzz]

Purpose

Moss-TTS streaming decode has a structural chunk-size cliff: initial_codec_chunk_frames: 1 emits a tiny first chunk (80 ms audio), then jumps directly to steady 15-frame chunks (1.2 s). Under C8 concurrency this causes audible underruns right after the first packet: the client's playback buffer never builds up before the pipeline settles into 1.2 s delivery quanta (measured: 0% streaming continuity, ~0.37 s mean underrun per request).

This PR ports the static chunk ramp (Phase 1 of the Qwen3-TTS ramp design, #5152) to the Moss-TTS raw async-chunk path: an opt-in codec_chunk_ramp: [2, 4, 8, 15] ladder replaces the cliff with gradual steps, reusing parse_chunk_ramp / ramp_chunk_size from chunk_size_utils.py unchanged.

The adaptive ramp (#6001 Phase 2 controller) is intentionally not part of this PR; it will be proposed separately for Moss-TTS once stage-1 chunk batching interacts cleanly with controller-chosen sizes.

Changes

• moss_tts.py (stage input processor): threshold selection becomes 2-way — static ramp ladder (indexed by the connector-maintained ramp_chunk_count) when codec_chunk_ramp is configured, otherwise the original IC/steady logic, byte-identical.
• modeling_moss_tts_codec.py: add module-level _resolve_streaming_graph_frame_sizes() built on the same parse_chunk_ramp helper the processor uses (plus a small _connector_extra_cfg accessor), so both consumers share one validated ladder: codec_chunk_ramp: left null in YAML, a malformed string, or a single-entry list disables the ramp on both sides (warn only) — no divergence, no raise at codec init. Ramp sizes join _streaming_graph_frame_sizes so every ladder (B, T) combination is captured during NPUGraph warmup — no runtime captures, no eager fallback.
• moss_tts_local.yaml: documented, commented-out codec_chunk_ramp example.
• New UT: tests/model_executor/stage_input_processors/test_moss_tts_chunk_ramp.py (11 cases: ladder sequence, finished flush, steady fallback past ladder, string-form config, IC/steady backward compat, empty input; codec-side resolver — absent key, explicit YAML null, invalid string / single-entry / non-positive values, valid list and comma-separated string).

Opt-in: with codec_chunk_ramp absent, the processor takes the original code path and the codec warmup set is unchanged ({initial, steady}). Zero impact on existing deployments.

Chunk-size effect (steady = 15):

before:  1 → 15 → 15 → 15 → ...      (15x cliff after first packet)
after:   2 → 4 → 8 → 15 → 15 → ...   (gradual, all sizes pre-captured)

Test Plan

• Unit tests: pytest tests/model_executor/stage_input_processors/test_moss_tts_chunk_ramp.py -v — 11 passed (run on the NPU host, python 3.12 / pytest 9.1.1)
• ruff format --check + ruff check clean
• NPU validation on Ascend 910B2C: C8 x 40 voice_clone streaming benchmark (vllm bench serve). Primary campaign: 6 consecutive rounds on the post-review head (content-identical to this PR, see revision note), round 1 (cold start) discarded, median of rounds 2-6 reported. An earlier 9-round campaign on the pre-review head (identical runtime behavior — the fixup is a parsing-only refactor) showed the same distributions: RTF median 0.26 (0.22-0.30), continuity median 60% (35-100%)
• NPUGraph warmup log confirms all ladder sizes pre-captured: (B,T) = {1,2,4,8,16,32,64} x {1,2,4,8,15} = 35 graphs; capture count stayed at exactly 35 through all 9 rounds of the pre-review campaign (zero runtime captures, zero graph-limit warnings, verified from service log); the fixup does not change the warmup set for a valid ladder (same parse_chunk_ramp result), and the post-review startup log shows the same 35-graph warmup
• Backward-compat: with codec_chunk_ramp commented out, the processor takes the byte-identical original code path (UT TestBackwardCompat) and the codec warmup set stays at the original {1,15} x batch buckets (14 graphs, as measured in ramp-off rounds on 2026-09-23)
• Audio correctness: streamed output complete, no missing frames / no bit-exact repeats (verify_audio.py incl. duplicate detection), listening check

Test Result

Environment: Ascend 910B2C, CANN 9.1.0, PyTorch 2.10.0, TorchNPU 2.10.0.post4, vLLM v0.28.0, vllm-omni = v0.28.0 + #7008 + #7280

Benchmark: C8 concurrency, voice_clone task, 40 requests per round, identical text/model/hardware across arms; only variable = codec_chunk_ramp. Baseline measured 2026-09-23 on the same server/benchmark (ramp off; the feature is opt-in and the ramp-off code path is unchanged).

Exact revision benchmarked (per review request): the primary campaign ran on a tree content-identical to this head — vLLM-Omni v0.28.0 (eb11446b) + a769d935 (#7008) + ae3880f0 (#7280) as the git-committed base, plus this PR's patches deployed by an anchor-patch tool and blob-hash verified against head. Our NPU stack (vllm-ascend) is pinned to vLLM 0.28.0, so the main-based head itself was not re-benchmarked. An earlier 9-round campaign ran on the pre-review head; the fixup (2a4826d) is a parsing-only refactor (identical behavior for a valid ladder — 11 CPU UTs; ramp-off path untouched), and both campaigns agree within round variance.

Metric	Baseline (ramp off)	Static ramp [2,4,8,15]	Delta
RTF (mean per round)	0.21-0.22	0.26 median (0.24-0.27)	+18%
Streaming continuity OK	0.0%	67.5% median (60-82.5%)	+67.5pp
Underrun mean (s)	~0.37	0.07 median (0.04-0.12)	-81%
TTFP mean (ms)	~1130	1153 median (1018-1246)	~neutral

Key findings:

• Streaming continuity 0% -> 67.5% median: the post-first-packet underrun window is eliminated for most chunks; the 1->15 cliff no longer starves client playback right after TTFP
• Underrun -81% (0.37s -> 0.07s median)
• RTF trade-off +18% (0.22 -> 0.26 median): the ladder adds ~2 extra codec decode calls per request during ramp phase; RTF remains far below 1.0 (real-time preserved with >3x headroom)
• TTFP neutral: chunk 0 grows 1->2 frames (+~40ms decode), within run-to-run variance
• Zero runtime NPUGraph captures: all 35 ladder (B,T) graphs are captured during warmup; the capture count stayed at exactly 35 across all rounds of the pre-review campaign (verified from service log), and the fixup does not change the warmup set for a valid ladder (same parse_chunk_ramp result)
• Run-to-run variance note: rounds correlate with request-arrival overlap (benchmark peak concurrency 12-16 vs configured 8); within both campaigns RTF/TTFP improve monotonically as arrival overlap rises. This is a known environment characteristic (the ramp-off baseline itself shows ±71% TTFP variance across repeated rounds) and is independent of this feature's code path, which is fully deterministic (fixed ladder, all graphs pre-captured)

Related

• feat(qwen3-tts): add codec_chunk_ramp for gradual chunk size warmup #5152 (Phase 1: static chunk ramp for Qwen3-TTS — utilities reused here)
• [Feature][TTS] adaptive chunk ramp (Phase 2 buffer-feedback controller) #6001 (Phase 2: adaptive ramp for Qwen3-TTS — follow-up for Moss-TTS)
• [Performance] Capture MOSS-TTS codec streaming decode with NPUGraph #7280 (Moss-TTS codec NPUGraph — this PR keeps all ladder sizes in its warmup capture set)
• [Bugfix][NPU] Fix talker_mtp ACL graph capture missing force_uniform_decode #7008 (force_uniform_decode bugfix — prerequisite for NPU serving)

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/model_integration.md, docs/design/module/ar_runtime.md.

Module owners: @tzhouam @fake0fan @Gaohan123

Routing: @tzhouam via module of the changed files, CODEOWNERS; @fake0fan via module of the changed files; @Gaohan123 via module of the changed files

@Wallbreazzz, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[Wallbreazzz]

Author self-review — re-read the full diff before submitting; checked:

1. Opt-in path is byte-identical: with codec_chunk_ramp absent, parse_chunk_ramp returns None and the processor takes the original 3-line IC/steady branch unchanged; _streaming_graph_frame_sizes collapses to {initial, steady} (warmup stays at 14 graphs, verified on 910B). TestBackwardCompat pins both.
2. Counter ownership: ramp_chunk_count is read-only in the processor; increment stays with the transfer adapter/mixin send_chunk after a successful put — same contract as Qwen3-TTS's talker2code2wav_async_chunk. UTs mimic adapter-side increments (no processor-side mutation).
3. No stranded frames: is_finished still flushes all remaining frames regardless of ladder position (emit_frames = pending if is_finished else threshold); pinned by test_ramp_finished_flush.
4. NPUGraph interaction (the reason ramp sizes join _streaming_graph_frame_sizes): every ladder (B,T) is captured at warmup — on 910B2C the capture count is exactly 35 ({1,2,4,8,16,32,64}×{1,2,4,8,15}) and stayed at 35 across all 9 benchmark rounds (zero runtime captures, zero graph-limit warnings, verified from service log). Without this, off-warmup ladder sizes would hit the wrapper's frame-size filter and fall back to eager.
5. Config parsing: _connector_list mirrors the existing _connector_int; YAML list and comma-separated string forms both covered (test_ramp_string_config). No changes to chunk_size_utils.py, connectors, or any framework-shared file.
6. Lint/tests: ruff format --check + ruff check clean; 6 CPU UTs pass on the NPU host (python 3.12 / pytest 9.1.1); streamed audio verified (zero >10ms gaps, chunk-boundary RMS 1.02-2.23, zero bit-exact repeats, L/R corr 1.0000).

Known trade-offs (intentional scope):

• RTF +18% median (0.22 → 0.26, still ≪ 1.0): the ladder adds ~2 decode calls per request during ramp phase.
• Adaptive controller ([Feature][TTS] adaptive chunk ramp (Phase 2 buffer-feedback controller) #6001-style) deliberately deferred to a follow-up: controller-chosen sizes off the pre-captured ladder interact with stage-1 T-grouped batching and need separate validation.
• Run-to-run variance correlates with request-arrival overlap (peak concurrency 12-16 vs configured 8); the ramp-off baseline shows the same environment characteristic (±71% TTFP across repeated rounds).

@vllm-omni-review-bot please run an automated review.

[linyueqian]

Thanks for porting the static ramp, the approach is clean: the ramp-off path is byte-identical, the ladder reuses parse_chunk_ramp / ramp_chunk_size on the processor side, and adding the ladder sizes to _streaming_graph_frame_sizes so every (B, T) is captured at warmup is the right call. One initialization defect inline. Two notes on the evidence: the numbers were measured on v0.28.0 (eb11446b) plus #7008 and #7280, so please record the exact ramp revision you benchmarked and whether it matches this head; and since this is a latency/UX change with an RTF cost (+18% in your table), a CUDA run of the same C8 voice-clone streaming comparison on current main would help show the continuity and underrun gains carry over beyond the 910B2C. Static read of the four files against merge-base 3b02e533; no PR code executed.

[linyueqian]

[important] This parser disagrees with the one the stage input processor uses. With codec_chunk_ramp: left empty (YAML null), the key is present, so this line iterates None and raises TypeError in the codec constructor, and Stage 1 never starts, while parse_chunk_ramp in chunk_size_utils.py treats None or an invalid value as "ramp disabled" and only warns. Malformed strings and one-entry ladders diverge the same way (the processor disables the ramp, the codec raises or warms up sizes nobody emits). Deriving the warmup sizes from parse_chunk_ramp(extra_cfg, steady=...) keeps both sides on one validated ladder; a codec-init test for an explicit null and an invalid value would lock that in.

[Wallbreazzz]

Fixed in 2a4826d: _connector_list removed; the warmup sizes now derive via _resolve_streaming_graph_frame_sizes() → parse_chunk_ramp, the same helper the processor uses, so null / invalid / single-entry values disable the ramp on both sides with a warning instead of raising in codec __init__. Pinned by 5 new CPU tests (explicit null, malformed string, single-entry, non-positive, valid list + comma-separated string). Extracted as a module-level pure function rather than an init-level test because the decoder __init__ loads the ~1.6B audio tokenizer.

[vllm-omni-review-bot]

Omni ReviewBot routing record

Assigned Strict under experiment vllm-omni-strict-5050-20260829.

[vllm-omni-review-bot]

Omni ReviewBot review

PR description

Moss-TTS Local streaming still used a 1-frame first codec packet and then jumped straight to 15-frame (1.2 s) chunks. This PR adds an opt-in codec_chunk_ramp ladder on the raw async-chunk processor so emit sizes can step 2 → 4 → 8 → 15, reusing the existing parse_chunk_ramp / ramp_chunk_size helpers, while the codec warmup set also includes those ladder T values so NPUGraph capture covers every steady (B, T) the ramp will emit. With the key absent, threshold selection stays on the original initial/steady path and the graph set stays {initial, steady}. The commented Local YAML example is documentation only; default deployments do not enable the ramp.

Change flow

flowchart TD
  A["[EXISTING] Stage-0 raw codec rows"]:::existing --> B["[CHANGED] talker2codec_raw_async_chunk threshold"]:::changed
  C["[EXISTING] parse_chunk_ramp / ramp_chunk_size"]:::existing --> B
  D["[CHANGED] moss_tts_local.yaml codec_chunk_ramp example"]:::changed --> B
  B --> E["[CHANGED] MossTTSCodecDecoder graph T set"]:::changed
  E --> F["[EXISTING] Streaming NPUGraph warmup"]:::existing
  G["[NEW] test_moss_tts_chunk_ramp.py"]:::new --> B
  classDef existing fill:#e5e7eb,stroke:#6b7280,color:#111827
  classDef changed fill:#fef3c7,stroke:#d97706,color:#451a03,stroke-width:2px
  classDef new fill:#dcfce7,stroke:#16a34a,color:#052e16,stroke-width:2px
  classDef removed fill:#fee2e2,stroke:#dc2626,color:#450a0a,stroke-width:2px

Loading

Findings

• [P1] Codec ramp parser crashes on null/invalid values the processor treats as off — vllm_omni/model_executor/models/moss_tts/modeling_moss_tts_codec.py:683
_connector_list treats a present codec_chunk_ramp key as an iterable of ints. If deploy YAML sets codec_chunk_ramp: (YAML null), name in extra_cfg is true and [int(x) for x in raw] iterates None during MossTTSCodecDecoder.__init__, so Stage 1 never starts. A malformed string hits int(...) on line 682 the same way. The processor side already calls parse_chunk_ramp(cfg, steady=chunk_frames), which the existing helper contract treats as ramp-disabled (warn only) for None / invalid / one-entry values — so the two consumers can disagree on the same extra map. Derive _streaming_graph_frame_sizes from parse_chunk_ramp(extra_cfg, steady=self._stream_chunk_frames) (then union with initial/steady) and add a codec-init test for explicit null and an invalid value. Default absent-key path is unchanged.
  Existing thread: #8124 (comment)

[vllm-omni-review-bot]

Omni ReviewBot: finding feedback

[p1] Codec ramp parser crashes on null/invalid values the processor treats as off — vllm_omni/model_executor/models/moss_tts/modeling_moss_tts_codec.py:683

_connector_list treats a present codec_chunk_ramp key as an iterable of ints. If deploy YAML sets codec_chunk_ramp: (YAML null), name in extra_cfg is true and [int(x) for x in raw] iterates None during MossTTSCodecDecoder.__init__, so Stage 1 never starts. A malformed string hits int(...) on line 682 the same way. The processor side already calls parse_chunk_ramp(cfg, steady=chunk_frames), which the existing helper contract treats as ramp-disabled (warn only) for None / invalid / one-entry values — so the two consumers can disagree on the same extra map. Derive _streaming_graph_frame_sizes from parse_chunk_ramp(extra_cfg, steady=self._stream_chunk_frames) (then union with initial/steady) and add a codec-init test for explicit null and an invalid value. Default absent-key path is unchanged.

This finding appears in the bot's COMMENT review, but GitHub could not place it as an inline diff comment. If you are the PR author and disagree, react 👎 to this comment. The disagreement will be shown to the maintainer; it does not approve or merge the PR.

[Wallbreazzz]

@linyueqian Thank you for the review — all three points addressed (fixup 2a4826d):

1. Parser divergence — fixed as suggested: _connector_list removed; the warmup set now derives from _resolve_streaming_graph_frame_sizes(), which calls the same parse_chunk_ramp the processor uses, so null / invalid / single-entry values disable the ramp on both sides (warn only, no raise in codec __init__). Pinned by 5 new CPU tests. The derivation is a module-level pure function rather than a full decoder instantiation because __init__ loads the ~1.6B-param audio tokenizer.

2. Exact revision benchmarked — recorded in Test Result: the primary campaign ran on a tree content-identical to this head (v0.28.0 eb11446b + a769d935 + ae3880f0, patches blob-hash verified). Post-fixup re-verification: 11/11 UT + 5 warm rounds (RTF median 0.26, continuity median 67.5%), statistically identical to the pre-review campaign.

3. CUDA run — I don't have access to CUDA hardware, so I can't produce GPU numbers myself. The feature is platform-neutral (pure-Python emit logic; the codec change only extends the frame-size set both the CUDA and NPU wrappers consume). If a GPU run is required before merge, I'd appreciate a maintainer with CUDA capacity running the same C8 comparison.

[linyueqian]

Thanks, the fix looks good. The codec now derives its warmup sizes through the same parse_chunk_ramp as the processor, so null or invalid ladders disable the ramp on both sides, and the new tests pin that. The revision note answers my other question. A CUDA run would still be nice to have, but it is not a blocker.

One optional follow-up: the codec's codec_chunk_frames default is 0 while the processor's default is 15. A ramp without an explicit codec_chunk_frames then logs a misleading ramp[-1] != 0 warning and skips warming up the 15-frame graph. Aligning the defaults would avoid that.