[Model] Add Ming-omni-tts dense 0.5B pipeline

docs/models/supported_models.md

@@ -60,6 +60,7 @@ th {
 | `Qwen3TTSForConditionalGeneration` | Qwen3-TTS-12Hz-1.7B-CustomVoice | `Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice` | ✅︎ | ✅︎ | ✅︎ | ✅︎ |
 | `Qwen3TTSForConditionalGeneration` | Qwen3-TTS-12Hz-1.7B-VoiceDesign | `Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign` | ✅︎ | ✅︎ | ✅︎ | ✅︎ |
 | `Qwen3TTSForConditionalGeneration` | Qwen3-TTS-12Hz-1.7B-Base | `Qwen/Qwen3-TTS-12Hz-0.6B-Base` | ✅︎ | ✅︎ | ✅︎ | ✅︎ |
+| `MingTTSForConditionalGeneration` | Ming-omni-tts dense 0.5B | `inclusionAI/Ming-omni-tts-0.5B` | ✅︎ | | | |
 | `GLMTTSForConditionalGeneration` | GLM-TTS | `zai-org/GLM-TTS` | ✅︎ | | | |
 | `NextStep11Pipeline` | NextStep-1.1 | `stepfun-ai/NextStep-1.1` | ✅︎ | ✅︎ | | ✅︎ |
 | `MiMoAudioModel` | MiMo-Audio-7B-Instruct | `XiaomiMiMo/MiMo-Audio-7B-Instruct` | ✅︎ | ✅︎ | | |

docs/user_guide/examples/offline_inference/ming_tts.md

@@ -0,0 +1,21 @@
+# Ming-omni-tts
+
+Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/text_to_speech/ming_tts>.
+
+For the TTS model hub, see
+[`examples/offline_inference/text_to_speech/README.md`](https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/text_to_speech/README.md).
+
+## Example materials
+
+??? abstract "README.md"
+    ``````md
+    --8<-- "examples/offline_inference/text_to_speech/ming_tts/README.md"
+    ``````
+??? abstract "end2end.py"
+    ``````py
+    --8<-- "examples/offline_inference/text_to_speech/ming_tts/end2end.py"
+    ``````
+??? abstract "cases.yaml"
+    ``````yaml
+    --8<-- "examples/offline_inference/text_to_speech/ming_tts/cases.yaml"
+    ``````

docs/user_guide/examples/online_serving/ming_tts.md

@@ -0,0 +1,25 @@
+# Ming-omni-tts
+
+Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/text_to_speech/ming_tts>.
+
+For the online TTS serving hub, see
+[`examples/online_serving/text_to_speech/README.md`](https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/text_to_speech/README.md).
+
+## Example materials
+
+??? abstract "README.md"
+    ``````md
+    --8<-- "examples/online_serving/text_to_speech/ming_tts/README.md"
+    ``````
+??? abstract "run_server.sh"
+    ``````sh
+    --8<-- "examples/online_serving/text_to_speech/ming_tts/run_server.sh"
+    ``````
+??? abstract "openai_speech_client.py"
+    ``````py
+    --8<-- "examples/online_serving/text_to_speech/ming_tts/openai_speech_client.py"
+    ``````
+??? abstract "run_curl.sh"
+    ``````sh
+    --8<-- "examples/online_serving/text_to_speech/ming_tts/run_curl.sh"
+    ``````

examples/offline_inference/text_to_speech/README.md

@@ -17,6 +17,7 @@ list of supported architectures across all modalities, see
 | CosyVoice3 | `FunAudioLLM/Fun-CosyVoice3-0.5B-2512` | 2 (talker + code2wav) | ✓ | ✓ | — | 24 kHz |
 | Fish Speech S2 Pro | `fishaudio/s2-pro` | dual-AR | ✓ | ✓ | — | 44.1 kHz |
 | GLM-TTS | `zai-org/GLM-TTS` | 2 (AR + DiT) | ✓ (required) | ✓ | — | 24 kHz |
+| Ming-omni-tts | `inclusionAI/Ming-omni-tts-0.5B` | 2 (AR + audio VAE) | ✓ | ✓ | style / IP / dialect / TTA / podcast | 44.1 kHz |
 | Ming-flash-omni-TTS | `Jonathan1909/Ming-flash-omni-2.0` | single (talker only) | — (caption-controlled) | — | style / IP / basic captions | 44.1 kHz |
 | MOSS-TTS-Nano | `OpenMOSS-Team/MOSS-TTS-Nano` | single (AR + codec) | ✓ (required) | ✓ | voice_clone, continuation | 48 kHz |
 | OmniVoice | `k2-fsa/OmniVoice` | 2 (gen + dec) | ✓ | — | voice design, language hint | 24 kHz |
@@ -159,6 +160,46 @@ Streaming requires `async_chunk: true` in the stage config.
 
 ---
 
+## Ming-omni-tts
+
+Dense 0.5B two-stage TTS pipeline (`AR + flow` + audio VAE) at 44.1 kHz. The example covers style, IP voice, music-only generation, text-to-audio events, emotion, dialect, zero-shot cloning, podcast, speech+BGM, and speech+environment-sound cases.
+
+### Quick start
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case style \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+### Voice cloning
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case zero_shot \
+    --ref-audio /path/to/reference.wav \
+    --ref-text "在此奉劝大家别乱打美白针。" \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+### Streaming
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case basic \
+    --ref-audio /path/to/reference.wav \
+    --streaming \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+### Notes
+- `style`, `ip`, `bgm`, and `tta` do not require reference audio.
+- Reference-audio cases use `--ref-audio`; `zero_shot` also requires `--ref-text`.
+- `podcast` uses multiple references via `--ref-audio-paths`.
+- Full case details live in [`ming_tts/README.md`](ming_tts/README.md).
+
+---
+
 ## Ming-flash-omni-TTS
 
 Standalone talker-only deployment of Ming-flash-omni-2.0 at 44.1 kHz. Voice is controlled through caption fields (`风格` / `IP` / `语速`/`基频`/`音量`) rather than reference audio.

examples/offline_inference/text_to_speech/ming_tts/README.md

@@ -0,0 +1,240 @@
+# Ming-omni-tts Offline Inference
+
+`end2end.py` runs Ming dense 0.5B end to end with vLLM-Omni. It uses the in-repo Ming prompt builder directly, so the example request shape matches the real integration instead of a simplified wrapper.
+
+## Files
+
+| File | Purpose |
+|---|---|
+| `end2end.py` | Driver: CLI, case loading, prompt construction, orchestration (~150 lines) |
+| `cases.yaml` | All 11 built-in case definitions (prompt, text, instruction, ref-audio flags, flow controls) |
+| `_runner.py` | Engine management and audio output (streaming + blocking paths; internal helper) |
+
+## Model Overview
+
+Ming dense 0.5B is exposed here as a two-stage offline pipeline:
+
+- **Stage 0**: Qwen2-based AR generation with Ming prompt formatting and inline flow controls
+- **Stage 1**: audio VAE decode to mono 44.1 kHz waveform
+
+The example supports both:
+
+- **Blocking eager** via `vllm_omni/deploy/ming_tts.yaml`
+- **Async chunk eager** via `vllm_omni/deploy/ming_tts.yaml` (default `async_chunk: true`)
+
+## Setup
+
+Install vLLM-Omni with the platform requirements for your accelerator:
+
+```bash
+uv pip install -e .
+```
+
+The Ming offline example does not require a separate upstream Ming package.
+Reference-audio cases use the repo dependencies for audio loading,
+resampling, and CampPlus speaker extraction, including `soundfile`,
+`torchaudio`, and `onnxruntime`.
+
+## Supported Cases
+
+These cases cover the upstream dense 0.5B cookbook surface that maps cleanly onto the current vLLM-Omni example:
+
+- `style`: zero-speaker style-conditioned speech
+- `ip`: zero-speaker IP voice generation
+- `bgm`: music-only generation
+- `tta`: text-to-audio event generation with FlowLoss controls
+- `emotion`: reference-audio speech with emotion control
+- `basic`: reference-audio speech with speed / pitch / volume control
+- `dialect`: reference-audio speech with dialect control
+- `zero_shot`: reference-audio cloning with explicit transcript
+- `podcast`: multi-reference dialogue generation with automatic speaker embedding extraction
+- `speech_bgm`: speech with background music conditioning
+- `speech_sound`: speech with environmental sound conditioning
+
+## Quick Start
+
+Run the zero-speaker style example:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case style \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+Run zero-shot cloning with a transcript:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case zero_shot \
+    --ref-audio /path/to/10002287-00000094.wav \
+    --ref-text "在此奉劝大家别乱打美白针。" \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+Run emotion-controlled speech:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case emotion \
+    --ref-audio /path/to/emotion_prompt.wav \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+Run podcast generation with two reference clips:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case podcast \
+    --ref-audio-paths /path/to/CTS-CN-F2F-2019-11-11-423-012-A.wav /path/to/CTS-CN-F2F-2019-11-11-423-012-B.wav \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+The script automatically extracts one 192-d speaker embedding per reference WAV using the Ming model's `campplus.onnx`.
+
+If you already have precomputed multi-speaker embeddings, you can override extraction with:
+
+```bash
+--speaker-embedding /path/to/podcast_speaker_embeddings.json
+```
+
+where the JSON is a list of speaker embeddings, one 192-d vector per speaker.
+
+Run text-to-audio event generation:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case tta \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+Use async_chunk streaming:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case basic \
+    --ref-audio /path/to/10002287-00000095.wav \
+    --streaming \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager
+```
+
+`--streaming` uses `AsyncOmni` and the async_chunk stage config. It currently
+supports one prompt per process invocation; use blocking mode for
+`--num-prompts > 1`.
+
+Collect runtime stats and a manifest:
+
+```bash
+python examples/offline_inference/text_to_speech/ming_tts/end2end.py \
+    --case style \
+    --deploy-config vllm_omni/deploy/ming_tts.yaml \
+    --enforce-eager \
+    --enable-stats \
+    --stats-log-file output_audio/ming_style_pipeline.log \
+    --metadata-json output_audio/ming_style_manifest.json
+```
+
+## Reference Fixtures
+
+The upstream Ming cookbook uses these public audio fixtures from `inclusionAI/Ming-omni-tts/data/wavs`:
+
+- `10002287-00000094.wav` for zero-shot cloning
+- `10002287-00000095.wav` for `basic`
+- `emotion_prompt.wav` for `emotion`
+- `yue_prompt.wav` for `dialect`
+- `00000309-00000300.wav` for `speech_bgm` and `speech_sound`
+- `CTS-CN-F2F-2019-11-11-423-012-A.wav` and `CTS-CN-F2F-2019-11-11-423-012-B.wav` for `podcast`
+
+## Validation Matrix
+
+The repo-facing example is intended to cover the same dense TTS workflows used
+by the local Ming validation script:
+
+| Case | Blocking `deploy/ming_tts.yaml` | Async chunk `deploy/ming_tts.yaml` | Extra inputs |
+|---|---:|---:|---|
+| `style` | Yes | Optional smoke test | none |
+| `ip` | Yes | Optional smoke test | none |
+| `bgm` | Yes | Optional smoke test | none |
+| `tta` | Yes | Optional smoke test | none |
+| `emotion` | Yes | Yes | `--ref-audio emotion_prompt.wav` |
+| `basic` | Yes | Yes | `--ref-audio 10002287-00000095.wav` |
+| `dialect` | Yes | Yes | `--ref-audio yue_prompt.wav` |
+| `zero_shot` | Yes | Yes | `--ref-audio 10002287-00000094.wav --ref-text ...` |
+| `podcast` | Yes | Yes | two `--ref-audio-paths` |
+| `speech_bgm` | Yes | Yes | `--ref-audio 00000309-00000300.wav` |
+| `speech_sound` | Yes | Yes | `--ref-audio 00000309-00000300.wav` |
+
+## Validated Outputs
+
+Validation on an L4 GPU completed the full blocking matrix and the default
+async_chunk matrix. Default async_chunk matched blocking output frame counts
+and Stage-1 patch counts for every case:
+
+| Case | Blocking frames / patches / sec | Async chunk frames / patches / sec |
+|---|---:|---:|
+| `style` | 409248 / 29 / 9.28 | 409248 / 29 / 9.28 |
+| `ip` | 183456 / 13 / 4.16 | 183456 / 13 / 4.16 |
+| `bgm` | 1326528 / 94 / 30.08 | 1326528 / 94 / 30.08 |
+| `tta` | 465696 / 33 / 10.56 | 465696 / 33 / 10.56 |
+| `emotion` | 324576 / 23 / 7.36 | 324576 / 23 / 7.36 |
+| `basic` | 211680 / 15 / 4.80 | 211680 / 15 / 4.80 |
+| `dialect` | 239904 / 17 / 5.44 | 239904 / 17 / 5.44 |
+| `zero_shot` | 409248 / 29 / 9.28 | 409248 / 29 / 9.28 |
+| `podcast` | 437472 / 31 / 9.92 | 437472 / 31 / 9.92 |
+| `speech_bgm` | 296352 / 21 / 6.72 | 296352 / 21 / 6.72 |
+| `speech_sound` | 352800 / 25 / 8.00 | 352800 / 25 / 8.00 |
+
+## Key Arguments
+
+| Argument | Description |
+|---|---|
+| `--model` | Hugging Face repo or local Ming checkpoint path |
+| `--deploy-config` | Deploy config YAML. Use `vllm_omni/deploy/ming_tts.yaml` |
+| `--case` | Built-in demo case |
+| `--ref-audio` | Single reference wav path for cloning-style cases |
+| `--ref-audio-paths` | Multiple reference wav paths, used by `podcast` |
+| `--ref-text` | Reference transcript. Required for `zero_shot` |
+| `--instructions` | Free-form Ming instruction string |
+| `--instruction-json` | Structured Ming instruction JSON |
+| `--speaker-embedding` | JSON file containing a 192-d speaker embedding |
+| `--extract-speaker-embeddings` | Force CampPlus speaker extraction from the provided reference audio paths |
+| `--max-decode-steps` | Override `ming_max_decode_steps` |
+| `--num-prompts` | Repeat the same case N times. Outputs are indexed when `N > 1` |
+| `--streaming` | Use `AsyncOmni` and async_chunk transport |
+| `--enforce-eager` | Recommended for Ming dense; non-eager is out of scope |
+| `--enable-stats` / `--log-stats` | Enable vLLM-Omni per-request stats logging |
+| `--stats-log-file` | Optional path for the stats log |
+| `--metadata-json` | Optional path for the run manifest JSON |
+| `--stage-init-timeout` | Per-stage initialization timeout in seconds |
+| `--init-timeout` | Total initialization timeout in seconds |
+| `--batch-timeout` | Batch timeout in seconds |
+| `--worker-backend` | `multi_process` or `ray` |
+| `--ray-address` | Ray cluster address when using `--worker-backend ray` |
+
+## Output
+
+- The script writes one mono 44.1 kHz WAV file per run
+- Default output directory: `output_audio/`
+- Default filename: `ming_<case>.wav`
+- When `--num-prompts > 1`, outputs are indexed as `ming_<case>_00000.wav`, `..._00001.wav`, etc.
+- When stats are enabled, the script can also write:
+  - a stats log file such as `ming_style_pipeline.log`
+  - a manifest JSON with per-output metadata, stage durations, peak memory info,
+    and streaming client latency metrics when `--streaming` is used
+
+## Notes
+
+- `style` and `ip` are zero-speaker paths and do not require a reference clip
+- `emotion`, `basic`, `dialect`, `speech_bgm`, and `speech_sound` require one reference clip
+- `zero_shot` requires both `--ref-audio` and `--ref-text`
+- `podcast` requires at least two reference clips via `--ref-audio-paths`
+- `podcast` automatically extracts one speaker embedding per reference clip
+- `--speaker-embedding` may contain either one 192-d vector or a list of 192-d vectors
+- `--enforce-eager` was used for the validated runs
+- Validation on the L4 GPU used SDPA for the Ming audio VAE instead of
+  FlashAttention2, which is the preferred default when available.