Skip to content

[Diffusion][HunyuanVideo] Fuse eager QKV packing and high-quality QKNorm - #34617

Merged
BBuf merged 1 commit into
sgl-project:mainfrom
BBuf:bbuf/b300-hunyuan-eager-fusions
Aug 13, 2026
Merged

BBuf merged 1 commit into
sgl-project:mainfrom
BBuf:bbuf/b300-hunyuan-eager-fusions

Conversation

@BBuf

@BBuf BBuf commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Reduce HunyuanVideo eager overhead with three scoped paths:

  • a lossless Triton kernel that applies image RoPE while packing image/text Q, K and V into contiguous attention inputs
  • a quality=high strided QK RMSNorm path that avoids input contiguous copies
  • the existing quality-gated fused linear+GELU epilogue for the Hunyuan MLP

The QKV/RoPE pack uses per-signature first-use bit-exact verification. The numerically relaxed RMSNorm and GELU paths are mounted only for quality=high. The cleanup keeps mounting all-or-nothing when the public FlashInfer CuTe QK RMSNorm entry point is unavailable, so no partial quality configuration leaks into the model.

B300 performance

HunyuanVideo, 848x480x65 frames, seed 42, 30 steps, native backend:

configuration denoise e2e vs latest-main eager
latest main eager 23.7185 s 26.7040 s baseline
PR lossless eager 22.8496 s 25.8321 s +3.66% / +3.26%
PR quality=high eager 22.3054 s 25.2885 s +5.96% / +5.30%
PR torch.compile 21.6219 s 23.7775 s +8.84% / +10.96%

quality=high is 2.38% faster in denoise than the PR's lossless path. It remains 3.16% behind compile in denoise and 6.35% behind end to end; the table intentionally reports that remaining gap rather than implying parity.

The first compile attempt hit the benchmark driver's 180-second warmup limit while compiling the VAE. A retry with the same generated cache and a 900-second limit completed successfully; compilation itself took about 11 minutes, while the displayed measurements exclude warmup.

Measured against main dbebc1deb42b00befa3d0de67265d7003994c1ad. The B300 validation tree is identical to this PR's final Git tree.

Output comparison

All videos use the same prompt, seed, resolution, frame count and step count.

Lossless path

The latest-main and PR lossless MP4 files are byte-identical (0345e7c0ace3ed8adeebbf967a068c58a2e757f6be9b58a1672f77056164a67a).

latest main eager (before) PR lossless eager (after)
latest-main frame sheet lossless PR frame sheet

quality=high path

The high-quality output versus the latest-main eager video measures PSNR 43.35 dB and SSIM 0.9851 over all 65 frames.

latest main eager PR quality=high eager
latest-main frame sheet quality-high PR frame sheet

Click either frame sheet to open the corresponding MP4.

Validation

  • B300 focused QKV/RoPE, mount/unmount and QK RMSNorm tests: 4 passed
  • exactness covers two production-relevant QKV shapes and the unmounted reference path
  • changed-file pre-commit suite passed on the final tree
  • weight cache cleanup removed 39.03 GiB after the model runs

CI States

Latest PR Test (Base): ❌ Run #31687604964
Latest PR Test (Extra): ✅ Run #31693045443

@github-actions github-actions Bot added diffusion SGLang Diffusion jit-kernel labels Aug 12, 2026
@BBuf BBuf added run-ci CI: run the baseline test suite on this PR run-ci-extra CI: also run the extra suite (requires run-ci) labels Aug 12, 2026
@BBuf
BBuf force-pushed the bbuf/b300-hunyuan-eager-fusions branch from 8a127e2 to 204e74a Compare August 12, 2026 18:59
@BBuf
BBuf force-pushed the bbuf/b300-hunyuan-eager-fusions branch from 204e74a to 7870ddb Compare August 13, 2026 09:39
@BBuf BBuf changed the title [Diffusion][HunyuanVideo] Fuse eager QKV RoPE packing [Diffusion][HunyuanVideo] Fuse eager QKV packing and high-quality QKNorm Aug 13, 2026
@BBuf BBuf added the mergeable label Aug 13, 2026
@BBuf
BBuf merged commit 3c1791a into sgl-project:main Aug 13, 2026
140 of 196 checks passed
saturn-acc pushed a commit to saturn-acc/sglang that referenced this pull request Aug 16, 2026
Atituiset pushed a commit to Atituiset/sglang that referenced this pull request Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion jit-kernel mergeable run-ci CI: run the baseline test suite on this PR run-ci-extra CI: also run the extra suite (requires run-ci)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant