[Benchmark] Universal TTS benchmark: Qwen3-TTS + VoxCPM2 with 3 task types (voice-clone/default/design)#2835

Merged

hsliuustc0106 merged 26 commits into

vllm-project:mainfrom

linyueqian:feat/universal-tts-benchmark

Apr 24, 2026

Commits on Apr 19, 2026

chore: start feat/universal-tts-benchmark off upstream/main
linyueqian
committed
feat(bench): add SeedTTSTextDataset for default-voice benchmarking
linyueqian
committed
feat(bench): add SeedTTSDesignDataset for voice-design benchmarking; clean up test stubs
linyueqian
committed
fix(bench): always set seed_tts_row for PCM capture; register seed-tts-text/design; update CLI choices
linyueqian
committed
feat(bench): add voice-design dataset (20 prompts); DFX runner excludes task/enabled/eval_phase metadata
linyueqian
committed
test(bench/dfx): add test_enabled_false_entry_is_skipped to runner metadata tests
linyueqian
committed
feat(bench): add universal TTS benchmark CLI (bench_tts.py) with model registry
linyueqian
committed
fix(test): correct benchmark_params access in test_enabled_false_entry_is_skipped
linyueqian
committed
feat(bench/dfx): add VoxCPM2 stage config + universal TTS DFX config (Qwen3-TTS + VoxCPM2, 3 tasks)
linyueqian
committed
fix(bench): use math.isnan, datetime import, remove dead request_rate param
linyueqian
committed
fix(bench/dfx): align num_prompts array length with max_concurrency in voice_design perf entry
linyueqian
committed
chore(bench): remove per-model TTS benchmarks; fix ruff style
linyueqian
committed
feat(bench/tts): restore plot_results.py and bench_voxcpm_offline.py
linyueqian
committed
fix(bench/tts): use vllm-omni binary instead of python -m vllm for bench serve
linyueqian
committed
fix(bench/tts): use 'instructions' key for VoiceDesign speech_extra
linyueqian
committed
fix(bench/tts): address review comments (P1, P2, P3, P4, P5)
linyueqian
committed
feat(bench/dfx): split CI test matrix into latency / throughput / quality regimes
linyueqian
committed
refactor(bench/tts): migrate to Pipeline + Deploy schema from #2383
linyueqian
committed
fix(tests/benchmarks): stop shadowing real vllm package in sibling tests
linyueqian
committed
fix(bench/dfx): bundle seed_tts_smoke + gate seed-tts voice_clone entries
linyueqian
committed
feat(bench/dfx): use median TTFP/RTF baselines for latency/throughput regimes
linyueqian
committed

Commits on Apr 22, 2026

test(bench/tts): add core_model/cpu marks, switch to pytest-mock mocker
linyueqian
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Benchmark] Universal TTS benchmark: Qwen3-TTS + VoxCPM2 with 3 task types (voice-clone/default/design)#2835

[Benchmark] Universal TTS benchmark: Qwen3-TTS + VoxCPM2 with 3 task types (voice-clone/default/design)#2835
hsliuustc0106 merged 26 commits into
vllm-project:mainfrom
linyueqian:feat/universal-tts-benchmark

Commits on Apr 19, 2026

chore: start feat/universal-tts-benchmark off upstream/main

feat(bench): add SeedTTSTextDataset for default-voice benchmarking

feat(bench): add SeedTTSDesignDataset for voice-design benchmarking; clean up test stubs

fix(bench): always set seed_tts_row for PCM capture; register seed-tts-text/design; update CLI choices

feat(bench): add voice-design dataset (20 prompts); DFX runner excludes task/enabled/eval_phase metadata

test(bench/dfx): add test_enabled_false_entry_is_skipped to runner metadata tests

feat(bench): add universal TTS benchmark CLI (bench_tts.py) with model registry

fix(test): correct benchmark_params access in test_enabled_false_entry_is_skipped

feat(bench/dfx): add VoxCPM2 stage config + universal TTS DFX config (Qwen3-TTS + VoxCPM2, 3 tasks)

fix(bench): use math.isnan, datetime import, remove dead request_rate param

fix(bench/dfx): align num_prompts array length with max_concurrency in voice_design perf entry

chore(bench): remove per-model TTS benchmarks; fix ruff style

feat(bench/tts): restore plot_results.py and bench_voxcpm_offline.py

fix(bench/tts): use vllm-omni binary instead of python -m vllm for bench serve

fix(bench/tts): use 'instructions' key for VoiceDesign speech_extra

fix(bench/tts): address review comments (P1, P2, P3, P4, P5)

feat(bench/dfx): split CI test matrix into latency / throughput / quality regimes

refactor(bench/tts): migrate to Pipeline + Deploy schema from #2383

fix(tests/benchmarks): stop shadowing real vllm package in sibling tests

fix(bench/dfx): bundle seed_tts_smoke + gate seed-tts voice_clone entries

feat(bench/dfx): use median TTFP/RTF baselines for latency/throughput regimes

Commits on Apr 20, 2026

docs(bench/tts): add README for universal TTS benchmark framework

feat(bench/dfx): add audio_throughput floor to throughput regimes

docs(bench/tts): add raw vllm bench serve commands for each task

docs(bench/tts): promote 'vllm bench serve --omni' to the primary path

Commits on Apr 22, 2026

test(bench/tts): add core_model/cpu marks, switch to pytest-mock mocker