Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
7807e8d
chore: start feat/universal-tts-benchmark off upstream/main
linyueqian Apr 16, 2026
6c4e751
feat(bench): add SeedTTSTextDataset for default-voice benchmarking
linyueqian Apr 16, 2026
878a69b
feat(bench): add SeedTTSDesignDataset for voice-design benchmarking; …
linyueqian Apr 16, 2026
30134f0
fix(bench): always set seed_tts_row for PCM capture; register seed-tt…
linyueqian Apr 16, 2026
74d6fe3
feat(bench): add voice-design dataset (20 prompts); DFX runner exclud…
linyueqian Apr 16, 2026
2dab511
test(bench/dfx): add test_enabled_false_entry_is_skipped to runner me…
linyueqian Apr 16, 2026
d661d08
feat(bench): add universal TTS benchmark CLI (bench_tts.py) with mode…
linyueqian Apr 16, 2026
5520229
fix(test): correct benchmark_params access in test_enabled_false_entr…
linyueqian Apr 16, 2026
c91a180
feat(bench/dfx): add VoxCPM2 stage config + universal TTS DFX config …
linyueqian Apr 16, 2026
3dec24d
fix(bench): use math.isnan, datetime import, remove dead request_rate…
linyueqian Apr 16, 2026
8092996
fix(bench/dfx): align num_prompts array length with max_concurrency i…
linyueqian Apr 16, 2026
6fe4b2d
chore(bench): remove per-model TTS benchmarks; fix ruff style
linyueqian Apr 16, 2026
ddf81c2
feat(bench/tts): restore plot_results.py and bench_voxcpm_offline.py
linyueqian Apr 16, 2026
d415746
fix(bench/tts): use vllm-omni binary instead of python -m vllm for be…
linyueqian Apr 16, 2026
6f5b9cc
fix(bench/tts): use 'instructions' key for VoiceDesign speech_extra
linyueqian Apr 16, 2026
f4562b3
fix(bench/tts): address review comments (P1, P2, P3, P4, P5)
linyueqian Apr 19, 2026
ba3d7c4
feat(bench/dfx): split CI test matrix into latency / throughput / qua…
linyueqian Apr 19, 2026
2ddf765
refactor(bench/tts): migrate to Pipeline + Deploy schema from #2383
linyueqian Apr 19, 2026
47fb2fd
fix(tests/benchmarks): stop shadowing real vllm package in sibling tests
linyueqian Apr 19, 2026
c05511f
fix(bench/dfx): bundle seed_tts_smoke + gate seed-tts voice_clone ent…
linyueqian Apr 19, 2026
18ebfa3
feat(bench/dfx): use median TTFP/RTF baselines for latency/throughput…
linyueqian Apr 19, 2026
798cea7
docs(bench/tts): add README for universal TTS benchmark framework
linyueqian Apr 20, 2026
a3819f5
feat(bench/dfx): add audio_throughput floor to throughput regimes
linyueqian Apr 20, 2026
eb30f55
docs(bench/tts): add raw vllm bench serve commands for each task
linyueqian Apr 20, 2026
969aead
docs(bench/tts): promote 'vllm bench serve --omni' to the primary path
linyueqian Apr 20, 2026
8f06be0
test(bench/tts): add core_model/cpu marks, switch to pytest-mock mocker
linyueqian Apr 22, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions benchmarks/build_dataset/seed_tts_design/en/meta.lst
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
vd001|||The quick brown fox jumps over the lazy dog.|A warm, friendly female voice with a slight American Midwest accent, speaking at a moderate pace with natural inflection.
vd002|||Welcome to the future of text-to-speech synthesis.|A deep, authoritative male news anchor voice, clear and professional with a measured cadence.
vd003|||The sunset painted the sky in brilliant shades of orange and pink.|A gentle elderly female voice, soft and wise, with a slight Southern American accent.
vd004|||Scientists have discovered a new species of deep-sea creature.|A young male voice with an Australian accent, curious and enthusiastic.
vd005|||Breaking news: a major climate summit opens today in Geneva.|A crisp female newsreader voice, neutral accent, confident and precise.
vd006|||In the beginning, there was darkness and silence across the universe.|A rich, dramatic bass male narrator voice, slow and deeply resonant.
vd007|||Come closer, I have something important to tell you.|A soft, intimate female voice, slightly whispery, warm and gentle.
vd008|||And they're off! The horses race toward the first turn at incredible speed.|An energetic male sports commentator, fast-paced and excited.
vd009|||Once upon a time, in a land far away, lived a very clever fox.|A light, playful voice with childlike enthusiasm, bright and clear.
vd010|||The ancient manuscript reveals secrets hidden for a thousand years.|A wise, measured elderly male voice, slow and deliberate, British English accent.
vd011|||Good evening, ladies and gentlemen, and welcome to our show.|A sophisticated female voice with a slight French accent speaking English, elegant and refined.
vd012|||System initialized. Running diagnostics. All systems nominal.|A clear, precise robotic-sounding voice, neutral and monotone with slight synthetic quality.
vd013|||I hear what you are saying, and it is completely understandable to feel that way.|A warm, empathetic female therapist voice, calm and reassuring, unhurried pace.
vd014|||Attention all units: proceed to grid reference seven-seven-alpha.|A firm, authoritative military male voice, clipped and commanding.
vd015|||Oh my goodness, you have to try this amazing new recipe I just found!|An enthusiastic, bubbly female voice, high energy and friendly.
vd016|||Dude, the waves were totally amazing out there today. Super happy about it!|A relaxed male voice with a California accent, casual and laid-back.
vd017|||The quarterly results exceed expectations across all major metrics.|A sharp, businesslike female voice, confident and efficient, fast-paced delivery.
vd018|||Chapter one. The morning sun filtered gently through the forest canopy.|A smooth, rich male audiobook narrator voice, expressive and engaging.
vd019|||To be or not to be, that is the question.|A theatrical female voice, dramatic and expressive, stage projection quality.
vd020|||And that is all for tonight. Stay well out there, everyone.|A warm, velvety male late-night radio DJ voice, smooth and intimate.
20 changes: 20 additions & 0 deletions benchmarks/build_dataset/seed_tts_smoke/en/meta.lst
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
smoke001|||The quick brown fox jumps over the lazy dog near the riverbank at sunset.
smoke002|||Welcome to the future of text-to-speech synthesis in production systems.
smoke003|||Yesterday the team finished rolling out the new authentication flow.
smoke004|||She walked carefully across the wet cobblestones, careful not to slip.
smoke005|||The conference call is scheduled for nine in the morning, Pacific time.
smoke006|||Please remember to save your work before closing the editor.
smoke007|||Two plus two equals four, but five hundred and forty three digits is long.
smoke008|||I would like a coffee with oat milk and a chocolate croissant please.
smoke009|||The library closes at eight on weekdays and six on Saturdays.
smoke010|||During the Renaissance, art and science flourished in European cities.
smoke011|||He whispered the secret word so quietly that no one else could hear.
smoke012|||Our flight departs from gate twenty three at eleven fifteen.
smoke013|||The storm knocked out power for six hours, but the backup generator kicked in.
smoke014|||Reading a good book on a rainy afternoon is one of life's great pleasures.
smoke015|||When the kettle whistled, she poured the hot water over the fresh tea leaves.
smoke016|||The algorithm runs in linear time, which is a big improvement over the previous approach.
smoke017|||In the distance, the mountains were shrouded in thick morning fog.
smoke018|||Our company reported record revenue for the fourth quarter of the fiscal year.
smoke019|||She explained the new policy in detail during the staff meeting this morning.
smoke020|||The children laughed and played in the garden until the sun began to set.
107 changes: 0 additions & 107 deletions benchmarks/qwen3-tts/README.md

This file was deleted.

Loading
Loading