[Sy0307]

Purpose

Add production-ready IndexTTS 2.5 support to vLLM-Omni using the official two-stage inference path:

• Stage 0: multilingual text frontend and autoregressive semantic-code generation.
• Stage 1: EnhancedCodec, S2Mel CFM/DiT, and BigVGAN synthesis at 22.05 kHz.
• OpenAI-compatible online serving and offline Omni inference.
• Zero-shot voice cloning and uploaded named-voice reuse.
• Official Chinese, English, Japanese, Spanish, and Arabic language preprocessing, plus an additional mixed Chinese/English (zhen) frontend mode.
• Text normalization and emotion control through vectors, text, or reference audio.
• Model-native speed control in the public speed range [0.5, 2.0].

The model payload and local asset contracts are fail-fast: missing conditioning, malformed model bundles, incompatible latent policy, and sequence-length mismatches return explicit errors instead of silent truncation, public-asset substitution, or zero audio.

Common-path changes are intentionally limited to explicit model/config/pipeline registration, tokenizer-free stage initialization, a model-scoped missing-HF-config path, native-speed adapter capability, the full-payload scheduler entry, and an IndexTTS-2.5-opted request-end payload snapshot. This PR does not include process environment locking, profiler changes, NPU changes, generic diffusion-attention changes, model-path sniffing, or the experimental latent deployment.

A recipe, online/offline examples, tests, and a Seed-TTS benchmark driver are included.

Test Plan

vLLM Version: 0.26.0

vLLM-Omni Commit: f515ee3f based on 6ae7ff78 (main)

Environment used for E2E validation:

• 1x NVIDIA H200
• Python 3.11
• PyTorch 2.11.0+cu128
• CUDA driver compatibility: 12.8
• IndexTTS 2.5 native model bundle

Validation performed:

• 489 focused pytest cases covering config/registry, engine setup, scheduler, OpenAI speech serving, adapters, tokenizer/text normalization, payload contracts, Stage 0, Stage 1, GPU runner, and request-end payload handling.
• Ruff lint and Ruff format checks over all changed Python files.
• Full HTTP /v1/audio/speech E2E with voice_01.wav reference voice, fixed seed 0, and native speeds 1.0 and 2.0.
• Whisper-small transcription of both generated clips.

Test Result

489 passed, 22 warnings in 28.42s
Ruff: All checks passed; 54 files already formatted

E2E input:

今天天气很好，我们一起去公园散步，感受温暖的阳光。

Speed	Duration	Sample rate	Whisper-small transcription
1.0	5.503 s	22050 Hz	今天天气很好我们一起去公园散步感受温暖的阳光
2.0	2.752 s	22050 Hz	今天天气很好我们一起去公园散步感受温暖的阳光

Both transcriptions exactly match the normalized input. The 2.0x duration is approximately 50% of the 1.0x duration, confirming that model-native speed control is applied once and produces the expected timing change.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[Sy0307]

Recipe quick start

Full recipe: recipes/IndexTeam/IndexTTS-2_5.md

1. Install

pip install -e '.[indextts2]'

Prepare the native IndexTTS 2.5 model bundle and a short, clean reference WAV for voice cloning.

2. Start the server

MODEL=/path/to/indextts-2.5

vllm serve "$MODEL" \
  --omni \
  --host 0.0.0.0 \
  --port 8092 \
  --trust-remote-code \
  --deploy-config vllm_omni/deploy/indextts2_5.yaml

3. Generate speech with a cloned voice

python examples/online_serving/text_to_speech/indextts2/speech_client.py \
  --api-base http://127.0.0.1:8092 \
  --model-version 2.5 \
  --model "$MODEL" \
  --ref-audio /path/to/reference.wav \
  --lang zh \
  --speed 1.0 \
  --text '你好，这是 IndexTTS 二点五语音合成测试。' \
  --output indextts2_5.wav

The same command supports:

• Native speed control: --speed 0.5 through --speed 2.0.
• Official model languages: --lang zh, en, ja, es, or ar.
• Additional frontend mode: --lang zhen for mixed Chinese/English text; this is not a separate model language.
• Cantonese (yue) is not listed as an officially supported IndexTTS-2.5 language.
• Emotion vector: --emo-vector 1 0 0 0 0 0 0 0 --emo-alpha 0.8.
• Emotion text: --use-emo-text --emo-text '开心、兴奋而且充满活力'.
• Emotion reference audio: --emo-audio /path/to/emotion.wav.
• Pre-normalized text: add --no-text-normalization.

4. Optional named-voice reuse

Upload the reference audio once:

curl -X POST http://127.0.0.1:8092/v1/audio/voices \
  -F 'audio_sample=@/path/to/reference.wav' \
  -F 'consent=user-consent-id' \
  -F 'name=indextts_demo_voice'

Then replace --ref-audio ... with:

--voice indextts_demo_voice

The validated output is 22.05 kHz mono WAV. H200 E2E covered voice cloning at 1.0x and 2.0x; Whisper-small matched the normalized input exactly for both files.

[linyueqian]

On the benchmarking side: does IndexTTS 2.5 need the dedicated benchmarks/tts/benchmark_indextts2_5.py, or could it be a model_configs.yaml entry instead?

The registry header states the intent directly:

# Universal TTS benchmark model registry.
# To add a new TTS model: add an entry here. No code changes required.

Every other TTS model in that directory (Qwen3-TTS, Higgs v3, VoxCPM2, the MOSS-TTS family) is registered that way, so this would be the first per-model driver script.

The README gives two reasons for the dedicated driver, and both look already covered by the shared harness:

1. The bundled emotion-model tokenizer: bench_tts.py:155 ends with cmd += extra_cli_args, fed by the argparse.REMAINDER at line 256, so --tokenizer <path> passes straight through to vllm bench serve.
2. IndexTTS specific extra_params: that is what task_extra_body is for, and Qwen3-TTS and MOSS already use it for their model specific request fields.

The shared path is also strictly more capable: --wer-eval for WER/SIM/UTMOS (line 242), audio_underrun in the default percentile metrics (line 124), and results that feed plot_results.py. A registry entry would look roughly like:

  IndexTeam/IndexTTS-2.5:
    supported_tasks: [voice_clone]
    backend: openai-audio-speech
    endpoint: /v1/audio/speech
    trust_remote_code: true
    task_extra_body:
      voice_clone:
        extra_params:
          lang: en
          text_normalization: true

One genuine gap I hit while checking this: bench_tts.py hardcodes --num-warmups 2 (lines 121 and 122) while your script exposes it as a flag. That seems better solved as an option on the shared harness so every model benefits, rather than as a reason to fork the driver.

One thing worth verifying rather than assuming: bench_tts.py resolves the model by its registry key, so serving from a local bundle path may need --served-model-name or a small key/path split. If that turns out to be awkward it is still likely a smaller change than maintaining a parallel driver plus its wrapper test.

The fixed n=500, concurrency 4/8/16/32 acceptance workload is worth keeping either way, just as a documented command in the README where the other models' reference workloads already live.

[hsliuustc0106]

Blocking on two items:

1. The new-model validation evidence is incomplete. The H200 smoke/Whisper result is useful, but this PR does not include a profiler trace/peak-VRAM analysis or a same-hardware baseline against an authoritative IndexTTS 2.5 runner. The recipe links the HF model and mentions num_beams=1, but does not identify a native runner/source revision or provide baseline results. Please add profiler top ops, peak VRAM and a repeated-inference leak check, plus native-vs-vLLM latency/RTF, VRAM, and output-quality results on identical inputs. If no authoritative 2.5 runner is public, document that and provide the profiling evidence at minimum.

2. Native speed is rejected for streaming. IndexTTS25Adapter declares native_speed_control and maps speed to duration_factor, but _validate_speech_streaming_request() rejects every speed != 1.0 before raw or SSE streaming reaches adapter preparation. Exempt native-speed adapters from this generic restriction and add raw+SSE coverage.

Validation: all changed CPU/core tests passed locally (459 passed, 22 warnings in 27.76s) with Python 3.12.13, Torch 2.11.0+cu130, and vLLM 0.26.0. Real-checkpoint E2E/profiling was not reproducible because the IndexTTS 2.5 bundle is unavailable in the review environment.

[Sy0307]

Thanks @linyueqian and @hsliuustc0106. I addressed the actionable code-review items in 11e49e89 (rebased onto the latest main):

Shared TTS benchmark harness

• Removed the dedicated benchmark_indextts2_5.py driver and its dedicated test.
• Registered IndexTeam/IndexTTS-2.5 in benchmarks/tts/model_configs.yaml.
• Kept the fixed acceptance sweep in the shared bench_tts.py workflow.
• Added generic --num-warmups, --served-model-name, and --request-seed options. The served-model override lets a registry key select defaults while the running server uses a local native bundle path.

Native speed with raw/SSE responses

• Moved the streaming speed decision out of the global request schema and into the serving layer, where it can consult the model adapter capability.
• Models without native speed control still reject streaming speed != 1.0.
• Native-speed adapters accept it, and IndexTTS maps it to duration_factor.
• Added endpoint coverage for both raw audio and SSE paths.

Focused validation on the H200 environment after the rebase: 236 passed across the shared benchmark CLI, speech serving, and TTS adapter test suites. Ruff format/check also pass locally.

I am still collecting the requested H200 profiler top-ops, peak-VRAM, and repeated-inference memory evidence. I will post those separately rather than treating the existing functional E2E result as sufficient performance validation. The official IndexTTS 2.5 native runner/checkpoint is not currently publicly reproducible, so any native-vs-vLLM comparison will be clearly scoped to an available pinned local bundle/source revision instead of presented as an authoritative upstream baseline.

[Sy0307]

Additional validation evidence

I collected the requested profiler and repeated-inference evidence on the current PR path. One correction to the earlier smoke-test wording: the remote host alias is H200, but nvidia-smi reports the physical device as NVIDIA L20X, 143771 MiB. I am therefore not presenting these results as H200 measurements.

Functional quality

Fixed-seed Chinese request:

今天天气很好，我们一起去公园散步，感受温暖的阳光。

Native speed	Output duration	Whisper-small transcript
1.0x	5.503 s	今天天气很好我们一起去公园散步感受温暖的阳光
2.0x	2.752 s	今天天气很好我们一起去公园散步感受温暖的阳光

Both outputs were 22,050 Hz mono WAV and matched the requested sentence after punctuation normalization.

Peak VRAM and repeated-inference leak check

The standard two-stage deploy config was used on one L20X GPU. NVML was sampled every 50 ms during a profiled request:

• peak process/device memory: 59,845 MiB
• peak sampled GPU utilization during the profiled request: 34%
• profiler-instrumented request latency: 19.224 s (profiler overhead; not a serving number)

The stage-0 startup accounting reported 3.57 GiB weights, 0.13 GiB peak activation, 0.22 GiB non-Torch memory, 0.06 GiB CUDA graph memory, and a 52.0 GiB configured KV-cache allocation. The reported 59,845 MiB is therefore the configured serving footprint, not a claim that the weights require 60 GiB.

After one warmup, 10 sequential unprofiled requests produced:

825, 772, 756, 732, 765, 771, 749, 750, 649, 767 ms

Median: 760.5 ms; min/max: 649/825 ms. NVML reported 59,845 MiB after every request, with no monotonic growth across the run.

PyTorch profiler top CUDA operators

Stage 0 (AR talker), excluding the inclusive execution-context annotations:

Operator/kernel	Self CUDA
NVJet GEMM 64x8_64x16...bias_TNT	38.973 ms
NVJet split-K GEMM	19.460 ms
NVJet GEMM 16x64_64x16...bias_TNN	16.989 ms
kernel_unified_attention	12.185 ms
compiled FX graph	12.402 ms
fused add + layer norm kernels	8.556 / 7.680 ms

Stage 1 (S2Mel + vocoder), excluding the inclusive execution-context annotations:

Operator/kernel	Self CUDA / CUDA total
aten::cudnn_convolution	20.283 / 22.855 ms
aten::cudnn_convolution_transpose	11.425 / 12.377 ms
aten::mm	10.686 / 11.145 ms
NCHW-to-NHWC kernels	8.129 ms
flash_attn::_flash_attn_forward	7.185 / 7.648 ms
aten::addmm	6.249 / 6.901 ms

The complete local artifacts include per-stage gzipped Kineto traces, profiler tables, memory snapshots, the 50 ms NVML sample log, and the repeat-run TSV.

Native-runner comparison boundary

I also rechecked the official public channels. The official index-tts/index-tts repository currently publishes download/runtime instructions for IndexTTS, IndexTTS-1.5, and IndexTTS-2, but not an authoritative IndexTTS-2.5 runner/source revision. The 2.5 HF endpoint is not publicly readable without authorization. We have a local 2.5 bundle and an unversioned reference-code snapshot, but using that as an “authoritative upstream baseline” would overstate its provenance.

For that reason I am providing the reproducible vLLM profiling, memory-stability, latency, and ASR evidence above, while explicitly not fabricating a native-vs-vLLM comparison until IndexTeam publishes or identifies a pinned public 2.5 runner.

[linyueqian]

lgtm