[diffusion] Restrict request-level quality to two validated tiers: lossless (default) and high by BBuf · Pull Request #33453 · sgl-project/sglang

BBuf · 2026-08-04T01:54:05Z

Motivation

From the maintainer discussion consensus:

The request-level --quality and --output-quality (output-file
compression) options are easy to confuse. Reducing quality to two
clearly documented levels with strict validation makes the sampling-side
switch harder to misread as an encoder setting.
We want one request-scoped opt-in for kernel/algorithm optimizations that
do not visibly degrade output but cannot pass the CI golden/ground-truth
bit-exact comparison. Every such fast path should gate on
quality == "high" instead of growing its own tier name.
Start with a single accelerated tier. Finer-grained control can be
reintroduced later without breaking the default.

Semantics

SamplingParams.quality: str = "lossless" — exactly two accepted values:

"lossless" (default): the exact reference path. Output is expected to be
bit-identical to the HF reference implementation and to pass the CI
golden/GT comparisons.
"high": opt into validated accelerated paths. Quality stays guaranteed —
the intent is to back every such path with mathematical acceptance
thresholds (e.g. PSNR > 25 against the reference; documented in the field
docstring, enforcement is follow-up work) — but output is no longer
bit-exact versus the HF reference or CI ground truth.

Anything else (including the previous medium/low tiers) fails request
validation with ValueError.

Changes

configs/sample/sampling_params.py: QUALITY_LEVELS = ("lossless", "high"), strict _validate membership check, CLI --quality uses
choices=["lossless", "high"].
MiniMax-H3 (the only current consumer):
- constants.py: the name-keyed profile dict is gone. quality="high"
  carries a single audited Cache-DiT parameter set,
  MINIMAX_H3_HIGH_QUALITY_CACHE_DIT_CONFIG = (4, 0.04, 1) (SSIM 0.931 /
  PSNR 28.16 dB vs lossless); quality="lossless" uses no Cache-DiT
  configuration.
- stages/denoising.py: MiniMaxH3DenoisingStage gates on
  quality == "high" and mounts/unmounts the audited Cache-DiT policy at
  the batch boundary.
- release_metadata.py / configs/pipeline_configs/minimax_h3.py:
  admission validates membership in QUALITY_LEVELS and triggers the
  strict validate_quality_deployment() check exactly when
  quality == "high".
- video_adapter.py: OpenAI video-API extra quality validated against
  the same two levels.
Docs: MiniMax-H3 cookbook quality section rewritten as the two-level
table (lossless baseline; high at 53.70 s, 1.40x, SSIM 0.931,
PSNR 28.16 dB); CLI reference updated.
Tests: minimal public-behavior coverage in test_sampling_params.py
(default lossless, explicit high accepted, invalid values —
medium/low/arbitrary strings/bools/numbers — rejected) plus the
existing admission/lowering contracts updated in
test_minimax_h3_admission.py. Tests that poked H3-internal constants
and mappings were removed.

Breaking change

quality accepts only "lossless" and "high". The previous
medium/low tiers are removed from the public API entirely and now fail
request validation; there is no compatibility shim. The old behavior
(more aggressive caching: 2.48x/2.91x speedup at SSIM 0.818/0.794) can be
reproduced with the process-wide SGLANG_CACHE_DIT_* environment
controls, which are unchanged.

Not in this PR (follow-ups)

Wiring any VAE/kernel fast path to this level (subsequent PRs gate their
conditions on quality == "high").
Mathematical acceptance thresholds (PSNR/SSIM gates) enforced in CI for
quality="high" paths.
--output-quality should eventually be renamed to emphasize that it is
output-file compression (e.g. --output-compression-quality) to finish
removing the quality vs output_quality confusion; intentionally left
out of this PR.

Validation

pytest on test_sampling_params.py, test_minimax_h3_admission.py, and
the related MiniMax-H3 / cache-dit / CLI / OpenAI-API unit files (H200).
Full multimodal_gen/test/unit sweep: the only failures reproduce
identically on clean origin/main on the same machine, i.e. pre-existing
and unrelated to this change.
Pinned pre-commit run --files ... fully green.

🤖 Generated with Claude Code

CI States

Latest PR Test (Base): ✅ Run #30871944582
Latest PR Test (Extra): ❌ Run #30872111814

gemini-code-assist · 2026-08-04T01:54:08Z

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

…ssless (default) and high quality: str = "lossless" (default) keeps the exact reference path whose output is bit-exact against the HF reference and CI golden comparisons; quality="high" opts into validated accelerated paths (quality guaranteed, intended to be backed by math thresholds such as PSNR>25, but not bit-exact). Any other value fails request validation; the previous medium/low tiers are removed with no compatibility shim. - SamplingParams: QUALITY_LEVELS=(lossless, high), strict validation; CLI --quality uses choices=[lossless, high] - MiniMax-H3: quality=high carries one audited Cache-DiT parameter set, MINIMAX_H3_HIGH_QUALITY_CACHE_DIT_CONFIG (SSIM 0.931 / PSNR 28.16 dB vs lossless); lossless uses no Cache-DiT configuration; admission validates the level and validate_quality_deployment triggers on quality == high. SGLANG_CACHE_DIT_* env controls unchanged - Docs: MiniMax-H3 cookbook quality section rewritten as the two-level table; CLI reference updated - Tests: minimal public-behavior coverage (default lossless, high accepted, invalid values rejected); internal-constant/mapping tests removed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…024^2 97.6->29.2 ms, 2048^2 437.2->168.5 ms) The FLUX.2 VAE decoder spends 97.6/437.2 ms per 1024^2/2048^2 image on H200: the fp32-upcast GroupNorm+SiLU chains dominate the profile and the NCHW decoder pays nchwToNhwc/nhwcToNchw transposes around every cuDNN conv. - New channels_last-only two-pass GroupNorm(+SiLU) Triton kernel (kernels/ops/diffusion/triton/group_norm_silu_twopass.py): fp32 statistics, affine folded into per-(batch, channel) scale/shift, native (N, H, W, C) / (N, L, C) rows support, optional SiLU epilogue; 4.8-25.9x vs aten fp32 at FLUX.2 decoder shapes. The existing general triton_group_norm_silu keeps serving the NCHW-contiguous HunyuanVAE / latent-upsampler paths unchanged. - FLUX.2 decoder fast paths (flux2_vae_cuda_opt.py), installed by the existing CudaPlatform.optimize_vae hook: channels_last decoding with the parameter layout swapped at decode entry, GroupNorm+SiLU fusion via the new kernel, nearest-2x upsample+Conv2d folded into ConvTranspose2d(k4,s2,p1), and a layout-safe single-head attention forward with the output projection folded into V. All rewrites are mathematically exact re-associations with lazy fp32 weight folding on first fast-path use; install is all-or-nothing and fail-closed. - Request-scoped gate wired in DecodingStage: quality == "high" (sgl-project#33453) runs the fast paths for the duration of the decode; the "lossless" default runs the original module path bit-for-bit. Validation (H200, frozen decode-input latents, steady-state mean of 5): - decode 1024^2 97.6 -> 29.2 ms (3.34x), 2048^2 437.2 -> 168.5 ms (2.59x); lossless path unregressed (97.1/437.0 ms) and byte-identical PNGs vs origin/main at both sizes; - fp32 ground truth (1024^2): fast-path bf16 error matches baseline bf16 error (mean abs 0.000828 vs 0.000824, p99 0.003051 vs 0.003027); - e2e sglang generate (FLUX.2-klein-4B): --quality high output PSNR 57.2 dB vs the default lossless output at the same seed; - 7 unit tests (kernel numerics/fallback contract, gate dispatch, lazy folding, fold math). Follow-ups land separately: exact chunked D512 attention composition for >=16k-token mid-blocks, and Qwen-Image VAE fast paths.

…tWan e2e 9.611 -> 9.125 s) Revives the sgl-project#30171 kernel behind the request-level quality gate. sgl-project#30171 was closed for a "consistency regression on B200": the fused kernel is numerically faithful (fp32 statistics, eager op boundaries preserved) but not bit-exact vs aten, so it moved the CI golden outputs. The quality tiers from sgl-project#33453 plus the sgl-project#33451 gate infrastructure remove that constraint: the kernel now runs only for requests that opt in with quality == "high"; the "lossless" default runs the original module path bit-for-bit (byte-identical harness outputs vs main). - New channels_last_3d Wan RMSNorm(+SiLU) Triton kernel (kernels/ops/diffusion/triton/wan_rmsnorm_silu.py): one program per (b, t, h, w) pixel channel row, fp32 statistics, WanRMS_norm eager op boundaries preserved per step. vs sgl-project#30171: supports the autocast dtype split (half activations + fp32 affine, output promoted to fp32 like aten) used by the bf16-decode Wan2.1 family; returns None on unsupported inputs (callers fall back); t/h/w are runtime args so the causal chunked decode does not recompile per chunk shape. - Wan decoder fast path (wan_vae_cuda_opt.py), installed by CudaPlatform.optimize_vae: FusedWanRMSNormSiLU wrappers on every residual-block norm1/norm2 + the decoder output head (29 sites on the FastWan2.2 VAE), dispatching on the shared sgl-project#33451 VaeFastPathGate; the DecodingStage wiring is reused unchanged. Install is all-or-nothing and fail-closed. Eager decodes only: under torch.compile Inductor already fuses the chain and the opaque custom op measures slightly slower (decode 4.90 -> 4.98 s), so compiled decodes keep the Inductor path for both tiers (verified at parity: 8.969 vs 8.948 s e2e, decode 4.90 both). Validation (H200 single GPU, seed 42): - kernel microbenchmark at captured production decoder shapes: 1.40-4.48x (fp32 FastWan 720p rows), 1.23-3.20x (bf16+fp32-affine Wan2.1 480p rows) vs the eager aten chain; - component harness (frozen production decode-input latents): FastWan 720p decode 5910 -> 5458 ms (-7.6%), Wan2.1 480p decode 1839 -> 1620 ms (-11.9%); gate-off byte-identical (sha256) to main and unregressed (5918 / 1833 ms on main); - e2e (one warmed process, sequential requests, median of 5 steady): FastWan2.2-TI2V-5B 720p x 81f eager 9.611 -> 9.125 s (-5.1%, DecodingStage 5.77 -> 5.11 s); 50-step models dilute the same decode win to ~1-2% e2e (50-step CFG denoise at the measured 0.315 s/step ~= 31.5 s vs the 0.66 s decode delta; a live 50-step Wan2.1-T2V run is currently blocked by the unrelated UMT5 qkv_proj loading regression on main); - quality=high vs lossless outputs: PSNR 36.1-39.7 dB (bar > 25), SSIM >= 0.956 over 3 prompts x 81 frames; fp32 ground-truth error of the fast path statistically matches the baseline (mean abs 0.001003 vs 0.000959); 8 unit tests. Stacked on sgl-project#33451 (pr-flux2-vae-fastpath); depends on it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ssless (default) and high (sgl-project#33453) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

BBuf requested review from AgainstEntropy, HaiShaw, JustinTong0323, mickqian, ping1jing2, sogalin, wisclmy0611, yichiche and zijiexia as code owners August 4, 2026 01:54

github-actions Bot added documentation Improvements or additions to documentation diffusion SGLang Diffusion labels Aug 4, 2026

BBuf added the run-ci CI: run the baseline test suite on this PR label Aug 4, 2026

BBuf force-pushed the pr0-quality-bool-refactor branch 2 times, most recently from 637d7e7 to 562236b Compare August 4, 2026 02:24

BBuf changed the title ~~[diffusion] Refactor request-level quality param from four-tier string to a single bool switch~~ [diffusion] Restrict request-level quality to two validated tiers: lossless (default) and high Aug 4, 2026

BBuf force-pushed the pr0-quality-bool-refactor branch from 562236b to 5b25310 Compare August 4, 2026 02:29

BBuf added run-ci CI: run the baseline test suite on this PR and removed run-ci CI: run the baseline test suite on this PR labels Aug 4, 2026

BBuf mentioned this pull request Aug 4, 2026

[diffusion] FLUX.2 VAE decoder fast path behind quality=high (H200: 1024^2 97.6->29.2 ms, 2048^2 437.2->168.5 ms) #33451

Merged

4 tasks done

BBuf merged commit c6f2a9c into sgl-project:main Aug 4, 2026
149 of 164 checks passed

BBuf mentioned this pull request Aug 4, 2026

[diffusion] Fuse DiT FFN tanh-GELU into up-proj GEMM (cublasLt epilogue) behind quality=high (Qwen-Image 1024^2 denoise 12.36 -> 12.05 s on H200) #33536

Merged

4 tasks done

BBuf mentioned this pull request Aug 4, 2026

[diffusion] Wan VAE RMSNorm+SiLU fusion behind quality=high (H200 FastWan2.2 e2e 9.611 -> 9.125 s) #33546

Merged

4 tasks done

patraxo mentioned this pull request Aug 5, 2026

[diffusion] MiniMax-H3 quality=high (Cache-DiT tier) topology-locked to 4xH200 — request 8xB300 audit #33720

Closed

saturn-acc pushed a commit to saturn-acc/sglang that referenced this pull request Aug 16, 2026

[diffusion] Restrict request-level quality to two validated tiers: lo…

329f545

…ssless (default) and high (sgl-project#33453) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

coolhok mentioned this pull request Sep 1, 2026

[diffusion] Extend MiniMax-H3 quality="high" SM100 B200 #37400

Closed

BBuf mentioned this pull request Sep 1, 2026

[diffusion] feat: TeaCache coefficient calibration #36767

Open

5 tasks done

Atituiset pushed a commit to Atituiset/sglang that referenced this pull request Sep 10, 2026

[diffusion] Restrict request-level quality to two validated tiers: lo…

48d84c1

…ssless (default) and high (sgl-project#33453) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Navigation Menu

Repository navigation

[diffusion] Restrict request-level quality to two validated tiers: lossless (default) and high - #33453

[diffusion] Restrict request-level quality to two validated tiers: lossless (default) and high#33453
BBuf merged 1 commit into
sgl-project:mainfrom
BBuf:pr0-quality-bool-refactor

BBuf commented Aug 4, 2026 •

edited by github-actions Bot

Loading

Uh oh!

gemini-code-assist Bot commented Aug 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

BBuf commented Aug 4, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Semantics

Changes

Breaking change

Not in this PR (follow-ups)

Validation

CI States

Uh oh!

gemini-code-assist Bot commented Aug 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

BBuf commented Aug 4, 2026 •

edited by github-actions Bot

Loading