Skip to content

Wan2.2 on the DiT scaffold: batched, and captured per served size - #1092

Draft
merceod wants to merge 4 commits into
users/atindra/pr-19-dit-attnfrom
users/atindra/pr-26-v2-wan-scaffold
Draft

merceod wants to merge 4 commits into
users/atindra/pr-19-dit-attnfrom
users/atindra/pr-26-v2-wan-scaffold

Conversation

@merceod

@merceod merceod commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #1085 (pr-19-dit-attn).

Why

Wan2.2 ran its own one-request loop on r9b, with no capture and no batching. The stock configs/wan22.yaml compiles only 1280x704x81, so the 832x480x33 clip the benchmarks request runs eager: 5.73 s a clip on t2v in window B8. Compiled, r9b does 4.05 s a clip in window H, and a second request adds little (t2v 14.83 clips/min at c=1, 15.47 at c=2).

What changed

  • mstar/model/wan22/submodules.py: Wan22DenoiseSubmodule on DenoiseLoopSubmodule is the new default (dit_loop: scaffold). Requests with the same grid, text length, CFG and mode batch, up to max_batch_size (default 2), at any mix of steps. Each compile_dit_sizes entry is compiled and captured at startup for T2V and I2V with CFG, at each capture_batch_sizes entry (default [1, 2]). The served config captures 8 graphs.
  • mstar/model/wan22/components/unipc.py: UniPC in a coefficient form. Its per-step branches fold into per-row weights staged once per request in one pinned copy, and it matches the reference solver to float32 rounding.
  • mstar/model/components/wan_vae.py: the VAE decode keeps the default layout on AMD GPUs, where MIOpen's channels-last conv3d is about 2x slower. NVIDIA stays channels-last.
  • configs/wan22.yaml serves 832x480x33 and lists 832x480 for the fast decode. dit_loop: legacy keeps the old loop.
  • This changes a default on every platform. On CUDA Wan2.2 now batches, and captures at startup, unless the config says otherwise. docs/models.rst covers the loop and the knobs.

Verification

  • CPU: ruff clean. Targeted r9b set 662 passed, 34 skipped. Media set 579 passed, 150 skipped, 1 failed, r9b 554, 148, 1, same id. CPU Core on DiT fusion on ROCm: compiled norms and the 1344x768 capture bucket #1091 + Wan2.2 on the DiT scaffold: batched, and captured per served size #1092 10406 tests, 37 failing, r9b 10379, 37, same ids. test_wan22_denoise_scaffold.py covers the coefficient form against an exact port and scaffold against legacy trajectories.
  • GPU, one MI325X: test_wan22_denoise_cuda_graph.py replays one captured step across a T2V and an I2V trajectory, and the media GPU set on the tip is 121 passed.
  • Window H, 8x MI325X, exclusive, four lanes side by side, two timed runs a cell, clips/min at 480x832x33, 20 steps. The harness gives every mstar arm compile_dit_sizes: [832x480x33], so r9b f906c7d compiles the served DiT here too. mstar-r9b-basefast adds the fast VAE decode. The tip f7122c3 is all of week 2, but this is the only Wan change outside shared host paths.
arm t2v c=1 t2v c=2 i2v c=1 i2v c=2 calibrated PSNR t2v, i2v
tip 15.91 16.57 15.82 16.49 20.47, 24.02 PASS
r9b 14.83 15.47 15.09 15.49 21.43, 24.76 PASS
r9b-basefast 14.95 15.44 14.87 15.22 21.43, 24.76 PASS
vllm-omni 14.84 15.32 14.56 15.04 9.24, 13.59 FAIL
  • Tip against r9b: t2v +7.3% and +7.1%, i2v +4.8% and +6.5%. Seconds a clip at c=1, t2v and i2v: tip 3.77, 3.79, r9b 4.05, 3.98, vllm-omni 4.04, 4.12. The calibrated floors are 18.49 and 22.2 dB. The tip reads about 1 dB under r9b on both sets, inside the cross-seed bar (median 0.037, worst 0.069). The same code moved up to 2 dB between launches in the branch's own A/B, so that reads as launch spread. The tables' raw column is the CUDA-derived gate.
  • On one GPU (shared node) the captured DiT forward is 143.9 ms against 141 ms served, and the VAE layout is about 0.17 s of the gain.

Not in this PR

  • On CUDA capture needs about 15.6 GiB (measured here for all 8 graphs, against a 13.9 GiB prediction). Whether that fits an 80 GB H100 beside the 1280x704x81 eager step is unchecked, and nothing here ran on NVIDIA. capture_batch_sizes: [1] or accelerator_graph: false are the way out, and dit_loop: legacy restores the old behaviour.
  • c=2 is +7%, under the +15% target, and only +4% over the tip's own c=1. The batch-2 forward was never timed alone, so whether batching or overlap with the other request's VAE is the gain is a hypothesis.
  • 1280x704x81 is captured at startup but was never served, timed or scored here.
  • Main reworked the Wan2.2 step after this was cut (bf70108: one-pass adaLN per frame, fused q/k norm+RoPE). It touches the same files and the two have not been combined or measured together.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant