[gcanlin]

1. Fast PCM conversion: convert float32 audio to interleaved PCM16 using in-place NumPy operations.
2. First-frame codec in Stage0: decode and send the first frame directly from Stage0; Stage1 maintains codec history and delivers the remaining audio.
3. First-frame MTP immediately after prefill: change prefill → wait for next scheduling step → first-frame MTP → first-frame codec → send to prefill → first-frame MTP → first-frame codec → send. Cache the codes and audio embedding for the next decode step, avoiding duplicate MTP computation while preserving output length and CUDA Graph execution.

H200, BF16, MPS, prefix cache enabled; full SeedTTS EN1088 voice cloning, output length 256. Two warm rounds per concurrency; cold round excluded. Main uses the previously measured base a46e9aae7.

Concurrency	main audio-s/s	PR audio-s/s	Throughput change	main mean TTFP (ms)	PR mean TTFP (ms)	TTFP change
C64	451.3	429.5	-4.8%	174.6	72.2	-58.7%
C128	514.3	527.3	+2.5%	561.6	177.8	-68.3%

File-reference benchmark (H200, 2026-10-09)

PR head ad135eedc, one H200 (GPU 6), BF16, MPS and prefix cache enabled; moss_tts_local_mrv2_optimized.yaml, with server CPU affinity 48–79. Full SeedTTS EN1088 per round, output length 256; all 3,264 requests succeeded, with no failures. The first C64 pass warms the reference caches; subsequent rows use the warmed server.

Run from this PR checkout, with the server allowing --allowed-local-media-path /root/vllm-omni-workspace:

PYTHONPATH="$PWD" \
VLLM_OMNI_BENCH_AUDIO_SAMPLE_RATE=48000 \
VLLM_OMNI_BENCH_AUDIO_CHANNELS=2 \
OMP_NUM_THREADS=1 \
taskset -c 80-95 /root/vllm-omni-workspace/.venv/bin/python benchmarks/tts/bench_tts.py \
  --host 127.0.0.1 \
  --port 8124 \
  --model OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5 \
  --task voice_clone \
  --locale en \
  --dataset-path /root/vllm-omni-workspace/seedtts_testset \
  --num-prompts 1088 \
  --concurrency 64 64 128 \
  --output-len 256 \
  --output-dir /root/vllm-omni-workspace/moss-pr8232-perf-20261008/artifacts/pr_file_ref_gpu6_20261009 \
  -- --seed-tts-file-ref-audio

Round	Concurrency	Reference-cache state	Audio throughput (audio-s/s)	Mean TTFP (ms)
1	C64	First pass	204.2	691.8
2	C64	After full-dataset warmup	442.5	60.1
3	C128	After full-dataset warmup	549.1	128.8

Reference audio is already stored on the server. --seed-tts-file-ref-audio sends a file:// URI instead of embedding the WAV as base64 in every request, reducing HTTP request-body handling and frontend queueing. These are separate measurements using file references; the original inline-audio results above are retained.

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/ar_runtime.md, docs/design/module/model_integration.md.

Module owners: @tzhouam @fake0fan @Gaohan123

Routing: @tzhouam via module of the changed files, semantic router, CODEOWNERS; @fake0fan via module of the changed files; @Gaohan123 via module of the changed files

@gcanlin, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[vllm-omni-review-bot]

Omni ReviewBot triage note

Automated triage of commit 4d114a308458 produced:

• Priority: high. Prompt maintainer attention is suggested.

These are automated triage suggestions only — the final decision belongs to the maintainers.

[gcanlin]

@Sy0307 I have reviewed the agent code. Most of them are clean now. PTAL.

[vllm-omni-review-bot]

Omni ReviewBot routing record

Assigned Strict on zcode (GLM-5.3-Flash) under experiment fleet-strict-cursor-grok46-zcode-glm53flash-5050-c5-z10-20261002.

[Sy0307]

Requesting changes for the E2E-confirmed stop-condition regression. The two code findings are attached inline.

Documentation still describes removed paths — recipes/OpenMOSS/MOSS-TTS.md:184 and lines 379–387.

The recipe still describes the below-32-active-stream fast path, Stage1-specific first-chunk graphs and gating, batch limit 16, and eager MTP as the default, although this PR removes those mechanisms or changes their defaults. These instructions lead users to tune behavior that no longer exists and omit the additional decoder resources introduced in Stage0. Please update both sections to describe the Stage0 first-frame path, its actual executor/TP/PP requirements, current defaults, and resource cost.

Validation scope: the stop finding was reproduced through the real speech HTTP endpoint on H200 using unmodified source, with the stop IDs configured in the Stage0 deployment YAML. The decoder finding was verified during actual single-GPU initialization with MultiprocExecutor; TP>1 and PP>1 were not tested. The documentation finding is based on code/documentation comparison.

[Sy0307]

With Stage0 stop_token_ids=[151656] and max_tokens>1, the first audio-slot token should terminate generation, but this path publishes audio before the stop decision is processed. H200 HTTP E2E tests returned an extra 80 ms of PCM in 15/15 requests on both ad135eed (C128) and 142a60db (capacity 16), while the pre-PR base produced no audio. Please gate publication on the canonical stop decision, or disable early publication for these constraints; audio already delivered to the client cannot be retracted.

[gcanlin]

Fixed in c448cd2

[Sy0307]

This initialization predicate does not match the sink-binding requirements: with Stage0 MultiprocExecutor, TP=1 and PP=1, the decoder is initialized even though _bind_first_audio_sink returns False. Actual H200 initialization confirmed approximately 1.98 GiB of decoder parameter weights, excluding buffers and graph memory, plus capture of 19 first-frame graph buckets. Please use a shared capability check for decoder initialization and sink binding so configurations without a sender do not allocate these unused resources.

[gcanlin]

Fixed in 4d114a3

[Sy0307]

LGTM and have checked all issues fixed.