Repository navigation

[Model] Add opt-in packed Flow and batched HiFT inference for CosyVoice3 - #8224

Merged

Sy0307 merged 38 commits into

vllm-project:mainfrom

Sy0307:perf/cosyvoice3-packed-inference

Sep 29, 2026

Merged

[Model] Add opt-in packed Flow and batched HiFT inference for CosyVoice3#8224
Sy0307 merged 38 commits into
vllm-project:mainfrom
Sy0307:perf/cosyvoice3-packed-inference

Commits on Sep 26, 2026
- [Core] Add reusable stage runtime and transport performance contracts
  Sy0307
  committed
  26e5383
- [Core] Preserve optional output rows and update scheduler test fixtures
  Sy0307
  committed
  776b4b1
Commits on Sep 27, 2026
- [Perf][CosyVoice3] Fold frozen HiFT generator weight norms after checkpoint load
  
  zhongc1211
  authored and
  Sy0307
  committed
  3a2f5c8
- [Bugfix] Keep folded CosyVoice3 HiFT weights usable by autograd
  
  zhongc1211
  authored and
  Sy0307
  committed
  7f661ed
- [Perf] Fold CosyVoice3 F0 weight norms after checkpoint loading
  
  zhongc1211
  authored and
  Sy0307
  committed
  fd40eb3
- [Core] Scope SHM wakeups and preserve streaming conditioning metadata
  Sy0307
  committed
  4988041
- [Model] Add opt-in packed inference profiles for CosyVoice3
  Sy0307
  committed
  59bf268
- [Core] Scope SHM wakeups and preserve streaming conditioning metadata
  Sy0307
  committed
  3883c60
- [Core] Align runtime inventory and prefix-cache type invariants
  Sy0307
  committed
  99ab585
- [Core] Expose shared nonblocking tensor copies for model adapters
  Sy0307
  committed
  71383f5
Commits on Sep 28, 2026
Commits on Sep 29, 2026