Skip to content

[Perf] qwen-image21: Pack BF16 gate/up projections on SM120 - #47090

Open
BBuf wants to merge 7 commits into
sgl-project:mainfrom
BBuf:codex/rtx6000-qwen21-bf16-gate-up
Open

BBuf wants to merge 7 commits into
sgl-project:mainfrom
BBuf:codex/rtx6000-qwen21-bf16-gate-up

Conversation

@BBuf

@BBuf BBuf commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Current-head update (2026-10-30): 188169fa0db0bb2d524d56a97184590f948cca07 passed the frozen 512 gates but failed the 1024 cleanup gate for T2I and editing. See the full revalidation and actual images below. The original report measures efcb88a2; it does not establish that the current head passed cleanup.

Qwen-Image 2.1 runs separate BF16 gate and up projections in its image MLP. Packing them into one projection reduces saved-request E2E latency by 3.16% for 512² T2I and 2.63% for 512² editing on RTX PRO 6000 Blackwell. The original 1024² cases improve by 1.08% and 0.94%; those smaller gains are retained below.

The change shares gate/up storage while preserving checkpoint parameter names and shapes, then reuses the existing packed SiLU-multiply operator. It adds no kernel or public flag. Eligibility is limited to SM120, resident unquantized TP1 projections, BF16 batch-one targets of 1024 or 4096 tokens, and quality="high". Prefixes, exact/lossless requests, LoRA wrappers, incompatible storage/layouts and torch.compile retain separate projections. Views are derived from parameters, with the existing _apply lifecycle repacking after CPU/GPU transfers.

Sources and comparison setup

  • Frozen main: af21e2b989e482cfdce39689126562f2209d53ac.
  • Final PR head: efcb88a201a41b1fecc516a60df0ca0a34fb6dd5.
  • Native vLLM-Omni: 817d4c17feb702289a2d2b2cbacfd594c262b7a7.
  • Checkpoint: Qwen/Qwen-Image-2.1@d26bb61231c349cf6b7896fa83353113880e1ba3; all 24 runtime files hash-verified.

The same physical GPU0 was used serially: RTX PRO 6000 Blackwell Server Edition, SM120, 97887 MiB, driver 595.84. GPU1 remained idle; the shared host has no exclusive lease, so ownership was checked at boundaries and during requests. Batch/concurrency 1, 50 steps, CFG 1, CPU seed 1101, identical prompts/reference image, empty negative prompt, BF16 native components with mixed-precision internals, no quantization.

SGLang uses native resident eager with cuDNN transformer attention, compile/BCG off. Omni retains its native regional compilation and decode graphs. Its default passes both 512 tasks and 1024 editing. Default 1024 T2I fails the Chinese-image gate (RGB PSNR 24.18 dB against SGLang cuDNN); the native compiled TORCHINDUCTOR_EMULATE_PRECISION_CASTS=1 variant passes and is used there. A separate valid-variant screen selected default for the other three cases. Failed variants remain archived. Equivalent native PNG compression is level 0: SGLang output_compression=0, Omni 100.

SGLang uses Torch 2.14.1+cu130 / Triton 3.8 / cuDNN 9.24; Omni uses Torch 2.13+cu130 / Triton 3.7.1 / cuDNN 9.20. Dependency versions are pinned in the evidence. These are measured native stacks, not identical compiler environments.

Final-head results

Primary E2E measures HTTP submission through receipt of complete response bytes; client decoding and artifact writes are outside the timer. Each task follows main/PR/Omni/Omni/PR/main, two fresh processes per arm. Each process first qualifies short, long, Chinese and repeated-short prompts. Then 512 uses 6 warmups + 8 measured requests; 1024 uses 3 + 6. The performance request is the frozen short prompt. Existing disk/compiler caches remain. Nothing was rerolled or filtered.

Case Arm Denoise p50 s¹ HTTP p50 / p95 s Sequential req/s² Peak sampled MiB³
512² T2I main 2.4262 2.4381 / 2.4412 0.410187 35257.4
512² T2I PR 2.3337 2.3608 / 2.3632 0.423580 35257.4
512² T2I Omni default 2.5093 2.5858 / 2.5903 0.386845 34267.4
512² edit main 2.5988 2.6530 / 2.6553 0.376981 35237.4
512² edit PR 2.5062 2.5825 / 2.5866 0.387163 35237.4
512² edit Omni default 3.3288 3.5839 / 3.6131 0.278668 38757.4
1024² T2I main 9.3668 9.4886 / 9.5053 0.105431 37493.4
1024² T2I PR 9.2774 9.3852 / 9.4043 0.106569 37517.4
1024² T2I Omni casts 10.2043 10.6764 / 10.7770 0.093976 39843.4
1024² edit main 10.8119 10.9692 / 10.9778 0.091196 43541.4
1024² edit PR 10.7068 10.8665 / 10.8799 0.092035 43541.4
1024² edit Omni default 11.8771 12.3454 / 12.3589 0.080992 44149.4

¹ Separate diagnostic runs, 4 measured requests per case/arm: SGLang synchronized native stages; Omni native pipeline timer plus a metadata-only observer. No Torch profiler during stage measurements. These scopes have instrumentation overhead and are excluded from primary ratios; do not subtract them from primary HTTP time. ² Reciprocal mean latency at concurrency 1, not saturation throughput. ³ NVML sampled every 250 ms; transient peaks can be missed. p95 is descriptive over 16 requests per arm at 512 and 12 at 1024.

Ratios below are geometric means of the two temporal pairs' per-process median ratios, rather than ratios of pooled p50s.

Case Main / PR E2E reduction vs main Omni / PR E2E reduction vs Omni
512² T2I 1.032652× 3.162% 1.095252× 8.697%
512² edit 1.027014× 2.630% 1.388124× 27.960%
1024² T2I 1.010966× 1.085% 1.132586× 11.706%
1024² edit 1.009452× 0.936% 1.136497× 12.010%

Main A/A endpoint drift was at most 0.089% at 512 and 0.020% at 1024. All final main/PR process CVs and main endpoint drift meet the original 0.5% limits. Omni fails that CV screen in one 512 edit process (0.621%) and both 1024 T2I processes (0.553% / 1.044%). Those failures and every sample remain recorded. Both pairs favor PR; even the fastest observed Omni request is slower than the slowest PR request in each case. The separate admission records explicitly use this post-screen interpretation, not a silent threshold change. Finite-sample separation is not a population guarantee.

Cached launch-to-ready ranges were 44.03–44.04 s for SGLang at 512, 52.04–54.04 s at 1024, and 36.02–40.02 s for Omni. This excludes the four qualification requests and explicit warmups, whose individual costs are recorded in the summary. Eager has no BCG capture cost. At 1024 each SGLang process warms only 1024: a preceding 512 warmup disables the pre-existing native packed-QKV path through its bit-exact gate in both main and PR. No unrelated QKV change is included.

Profile, cleanup and compatibility

Final representative traces show 64 separate gate/up GEMMs → 32 packed GEMMs, plus 32 existing packed SiLU calls per denoise step. At 512 total GPU kernels fall 480 → 448; fresh 1024-only traces retain native packed QKV and fall 416 → 384. Matched complete GPU step durations, diagnostic only:

Case Main ms PR ms
512² T2I 49.466 47.513
512² edit 51.971 50.165
1024² T2I 188.065 186.076
1024² edit 212.939 210.698

Cleanup narrows the original broader length predicate to the two model-qualified targets and simplifies comments/docs. Two fresh-process ABBA screens retained T2I pairs at +0.1338% and +0.1688%, exceeding the fixed 0.1% cleanup screen. These are failed screens, not discarded warmups. Complete production AST comparison finds only the length predicate changed; it is equivalent at the tested shapes. A separately preregistered within-process diagnostic alternated the actual archived precleanup/final functions at request boundaries: 8 ABBA quartets per 512 task and 6 per 1024 task, after 8 warmups. All 144 outputs match main. Final/pre ratios are 0.999757 / 1.000049 at 512 and 1.000071 / 1.000069 at 1024; each descriptive per-case t95 upper bound is below 1.001. This supports no measurable direct cleanup penalty within that bound, while retaining the inconclusive fresh-process evidence. It does not prove zero effect or provide a familywise guarantee. These instrumented timings never enter the performance table.

Final compatibility validation covers 68 saved requests, including 12 separate profiles: high/lossless/exact/high-return, both shapes, short/long/Chinese/repeated prompts. High T2I executes one prefill plus 49 BCG replays; each profiled replay contains 32 packed SiLU calls and 33 graph launches. No serving capture occurs. Editing exceeds the native conditioning bucket and remains eager; non-high requests also remain eager and restore main pixels with no packed-MLP calls. The two-shape warmup creates 10 actual graph entries. A single cached diagnostic observed 47480 vs 39782 MiB before requests and 58.04 vs 56.04 s ready time for BCG/eager. This is compatibility and cost evidence, not a BCG performance win.

Quality and real outputs

Every final saved SGLang qualification/measurement output has RGBA pixels byte-identical to its corresponding main reference. High PR/main uses fixed SSIM 0.99 / PSNR 40 dB gates; cross-framework RGB and alpha use 0.95 / 28 dB separately. Minimum final selected-Omni metrics:

Case RGB SSIM / PSNR dB Alpha SSIM / PSNR dB
512² T2I 0.991974 / 38.659 0.999529 / 63.664
512² edit 0.968473 / 32.031 0.999083 / 60.656
1024² T2I 0.972529 / 29.235 0.999029 / 60.492
1024² edit 0.997444 / 51.742 0.999331 / 62.145

CPU processor inputs, token IDs, reference pixels and initial noise were checked. Effective BF16 model timesteps match at 32 recorded boundaries; the native sigma schedule differs by at most two FP32 ULP from numpy.exp vs math.exp. This is not a bound on all solver arithmetic. T2I encoder hidden states match; edit hidden states differ before collapse. The runtime precision audit retains those differences. Final Omni native-stage observations verify BF16 scheduler input/output boundaries with autocast off. Cross-framework intermediate identity is not claimed.

Each panel below contains three actual output pairs (short, long, Chinese), main left / final PR right. Original RGBA pixels are kept at 1:1 with labels outside them; GitHub may scale the display. The source hashes, inputs and visual review are archived. Editing changes the reference red teapot to blue. These are generated results, not illustrations.

Qwen-Image 2.1 512 square t2i: main left, final PR right

Qwen-Image 2.1 512 square edit: main left, final PR right

Qwen-Image 2.1 1024 square t2i: main left, final PR right

Qwen-Image 2.1 1024 square edit: main left, final PR right

Validation, reproduction and limits

The real SM120 regression passed both 1024/4096-token cases: numeric/dispatch checks, original-path restoration, prefix fallback, independent storage, CPU/GPU round trip, actual dynamic LoRA weights and torch.compile(backend="eager", fullgraph=True) fallback. The latter checks frontend fallback, not full-model Inductor performance. Ruff 0.15.1/isort, Python/diff checks and Mint validation/broken-links passed. Final diff: 3 files, 114 additions / 7 deletions, including the regression and scoped docs.

Existing INT8 packing #44196 remains unchanged. Closed, unmerged #40918 is BF16/QKV/VAE prior art; this PR supplies a scoped BF16 path with parameter/lifecycle preservation and fresh SM120 model evidence. Latest main checked was 5cc9c933fc8acdffb5fc847d3b4824f082cd3297: its Qwen encoder is unchanged; the DiT differs from frozen main only in the reviewed BCG attention-output allocation. Three-way textual merge checks pass for the three changed files; this is not a new benchmark or runtime test of merged main.

Use separate pinned environments with /task containing the archived inputs/scripts, verified checkpoint at /task/cache/common/Qwen-Image-2.1, main at /task/repos/sglang, final at /task/candidates/qwen21-bf16-gate-up-v4, and Omni at /task/repos/vllm-omni. Start one server at a time and stop only its recorded owned process before the next arm. Example 512 launches:

export QWEN_CANDIDATE_DIFF_SHA=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
# Main SGLang environment:
SGL_REPO=/task/repos/sglang sh /task/launch_sgl_qwen21_main_512.sh
# Final PR, separate process:
SGL_REPO=/task/candidates/qwen21-bf16-gate-up-v4 sh /task/launch_sgl_qwen21_v4_512.sh
# Native Omni environment:
sh /task/launch_omni_qwen21_base_native.sh

Against the running PR server, qualify first with a fresh output label:

python /task/submit_qwen21_candidate_qualification.py sgl \
  --label reproduction-pr-t2i512 --quality high \
  --contract /task/inputs/qwen21-base-t2i-512-workloads.json \
  --repo /task/candidates/qwen21-bf16-gate-up-v4 \
  --expected-sha efcb88a201a41b1fecc516a60df0ca0a34fb6dd5

Then use bench_qwen21_serving.py with the same arm/contract/repo/SHA, --quality high --warmups 6 --repeats 8, a fresh --run-id, and --allowed-gpu-pids containing only the current owned server's NVML-visible PID(s). Recreate the A/A and paired orders. For 1024 use the original non-512 contracts, launch_sgl_qwen21_candidate.sh for main, launch_sgl_qwen21_v4_1024.sh for PR, --warmups 3 --repeats 6, and the native casts launcher for Omni T2I only. Every exact executed command and request is archived.

Controllers contain original host/container paths and reject overwritten output IDs; adapt them to a fresh namespace, recreate quality references, and never reuse recorded PIDs. Primary runs use SYNC_STAGE_PROFILING=0; separate denoise launchers use synchronized native stages. Profile requests are diagnostic only.

Performance claims cover these single-GPU, single-reference, batch/concurrency-one shapes. Other lengths, GPUs, quantized checkpoints, full-model offload/LoRA/compile combinations, broader caches and multi-GPU cases are not validated performance claims. The broader frozen model matrix remains incomplete.

Evidence: final numbers and warmup costs, cleanup admission and failed screens, runtime precision, paired plans, raw results and exact commands, actual traces, reproduction scripts.

Initial CI (before the ready-for-review update): runtime workflows stopped at the explicit Draft gate. The initial all-file lint job fails Ruff F821 (MLATokenToKVPool) in python/sglang/srt/managers/cache_controller.py:885. That file has the same blob eb71dc2709c1ede1c0060bb082731833c405dc9d in base 286e0185 and the tested merge 8471aa33; it is outside this three-file PR. Other pre-commit hooks pass, later docs checks are skipped. Local scoped validation is reported above; that initial CI run was not green.

Cleanup validation (2026-10-29)

Cleanup at 188169fa0: renamed the existing packed-BF16 dispatch flags and made their call sites explicit; packing eligibility, arithmetic, and kernels are unchanged. On RTX PRO 6000 Blackwell (SM120), PyTorch 2.14.1+cu130, the Qwen-Image 2.1 non-VAE fast-path selection passed 8 tests, including both MLP token sizes, restoration, offload, compile fallback, and dynamic LoRA. Changed-file pre-commit checks pass. The model timings and images above remain measurements of the previously documented head; this cleanup did not repeat full-model benchmarks.

Current-head revalidation: 1024 cleanup gate failed

Current head 188169fa0db0bb2d524d56a97184590f948cca07 passes the frozen 512 gates but fails the frozen 1024 cleanup gate in both T2I and editing. All samples and original thresholds are retained. This report does not qualify the current head. The original PR report remains historical evidence for efcb88a201a41b1fecc516a60df0ca0a34fb6dd5.

Frozen SGLang main af21e2b989e482cfdce39689126562f2209d53ac; native vLLM-Omni 817d4c17feb702289a2d2b2cbacfd594c262b7a7; checkpoint Qwen/Qwen-Image-2.1@d26bb61231c349cf6b7896fa83353113880e1ba3, all 24 runtime files hash verified. Same physical RTX PRO 6000 Blackwell Server GPU0 (GPU-1565b45e-a702-2015-9776-61f68c36fd59), GPU1 idle, driver 595.84. SGLang Torch2.14.1+cu130, Omni Torch2.13.0+cu130. BF16 native components with recorded internal mixed precision, CPU seed1101, 50steps, CFG1, batch/concurrency1, fixed inputs and RGBA PNG semantics.

New primary runs use native resident SGLang eager, SYNC0, without a profiler or observer. Omni retains its native regional compilation and decode graphs. 512 uses native default; 1024 T2I uses the previously quality-qualified native emulate_precision_casts, while 1024 editing uses native default. The earlier default1024 T2I quality failure remains in the original PR evidence; it is not removed or counted as a win.

Each resolution has a fresh two-process main A/A followed by the frozen paired primary schedule. Per task: two main and two native Omni processes (16 measured requests each), four original and four current processes (32 each); each process/task also retains four qualification requests, six excluded warmups and eight measured short requests. The second half reverses task order. Total primary: 12 processes/432 PNGs at512 and14 processes/432 PNGs at1024. A/A adds72 PNGs per resolution. Process pairs, not individual requests, are the independent units in paired confidence intervals.

Primary E2E and memory

Shape/task Arm n Mean s p50 s p95 s Serial response/s Peak NVML MiB
512 t2i main 16 2.435254704 2.435507814 2.438482404 0.410635 35257
512 t2i original 32 2.364894054 2.366075612 2.372967312 0.422852 35257
512 t2i current 32 2.364191113 2.364764111 2.371975307 0.422978 35257
512 t2i omni_default 16 2.586238788 2.585694485 2.591539777 0.386662 34269
512 edit main 16 2.657522689 2.656551864 2.662917140 0.376290 35237
512 edit original 32 2.578895797 2.578749324 2.584933220 0.387763 35237
512 edit current 32 2.578449149 2.579151311 2.584391232 0.387830 35237
512 edit omni_default 16 3.600092034 3.599287455 3.618790562 0.277771 38757
1024 t2i main 16 9.501570366 9.500171912 9.514016596 0.105246 37493
1024 t2i original 32 9.402412370 9.401856614 9.417416885 0.106356 37517
1024 t2i current 32 9.408172547 9.410254172 9.417993402 0.106291 37517
1024 t2i omni_casts 16 10.516653919 10.515226459 10.534325720 0.095087 39843
1024 edit main 16 10.977237993 10.978408742 10.985958670 0.091098 43541
1024 edit original 32 10.873791575 10.871960921 10.885918304 0.091964 43541
1024 edit current 32 10.878866522 10.879124714 10.890679547 0.091921 43541
1024 edit omni_default 16 12.363896572 12.365365414 12.374552425 0.080881 44149

Serial response rate is reciprocal mean serial latency, not a saturation-throughput result. Peak NVML includes process/device residency; it is not tensor allocator size.

Fixed gates and uncertainty

Shape/task E2E reduction vs main Omni/current mean ratio Cleanup mean change Fresh AA noise Cleanup t95 upper regression Primary / cleanup
512 t2i 2.918117% 1.093921204x -0.029723959% 0.163928012% 0.021939603% pass / pass
512 edit 2.975461% 1.396223786x -0.017319351% 0.255535351% 0.056332464% pass / pass
1024 t2i 0.982972% 1.117821114x +0.061262760% 0.017214899% 0.215976070% pass / FAIL
1024 edit 0.896140% 1.136505954x +0.046671365% 0.020528947% 0.095566251% pass / FAIL

Cleanup requires mean regression no greater than fresh AA noise AND a four-pair log-t95 regression upper bound below0.1%. Both1024 means exceed AA noise; T2I also fails the upper-bound test. The two1024 cleanup intervals include1: this does not prove a nonzero population regression or a source-code cause, and does not satisfy the frozen no-regression assurance. No unchanged primary rerun or relaxed gate was used.

Shape/task Ratio Process pairs Geomean Paired log-t95 interval
512 t2i main_over_current 2 1.030205214 [0.990343236, 1.071671663]
512 t2i omni_over_current 2 1.094071543 [1.051070823, 1.138831480]
512 t2i original_over_current 4 1.000250745 [0.999780652, 1.000721060]
512 edit main_over_current 2 1.031055015 [0.994291071, 1.069178308]
512 edit omni_over_current 2 1.396808971 [1.302986223, 1.497387515]
512 edit original_over_current 4 1.000283287 [0.999436993, 1.001130298]
1024 t2i main_over_current 2 1.010063336 [1.005114707, 1.015036330]
1024 t2i omni_over_current 2 1.117816311 [1.113326745, 1.122323981]
1024 t2i original_over_current 4 0.999415447 [0.997844894, 1.000988471]
1024 edit main_over_current 2 1.009509661 [1.007089446, 1.011935692]
1024 edit omni_over_current 2 1.137004190 [1.128434238, 1.145639227]
1024 edit original_over_current 4 0.999553357 [0.999045250, 1.000061722]

Both512 main/current intervals include1 despite their frozen gates passing; only two process pairs support these intervals. Do not claim all main/current intervals exclude1. Both1024 main/current and all four Omni/current intervals exceed1. Independent NumPy/SciPy reviews recomputed all statistics from actual raw results without the admission calculator.

Startup and excluded warmups

Shape/task Arm Launch-to-ready range s Sum of six warmup requests per process, range s
512 t2i main 44.031–44.044 14.635–14.640
512 t2i original 44.029–54.046 14.099–14.194
512 t2i current 44.032–52.042 14.095–14.188
512 t2i omni_default 34.016–36.022 15.517–15.545
512 edit main 44.031–44.044 15.963–15.986
512 edit original 44.029–54.046 15.403–15.470
512 edit current 44.032–52.042 15.401–15.470
512 edit omni_default 34.016–36.022 21.569–21.656
1024 t2i main 52.042–52.052 56.595–56.742
1024 t2i original 52.042–52.048 55.839–56.263
1024 t2i current 52.043–52.047 56.113–56.270
1024 t2i omni_casts 40.016–40.018 62.842–62.977
1024 edit main 52.042–52.052 65.562–65.633
1024 edit original 52.042–52.048 64.822–65.070
1024 edit current 52.043–52.047 64.887–65.102
1024 edit omni_default 36.017–36.020 73.911–74.011

Readiness includes native startup/warmup and, for Omni, native compilation/graph setup; these costs are not included in primary saved-request E2E. The same server may handle both tasks; per-task startup entries are not additive.

Quality

The twelve displayed fresh main/current primary image pairs have identical PNG bytes and RGBA pixels (RGB/alpha SSIM1, PSNR∞). Three distinct prompts per task/shape include short, long and Chinese text; repeated short requests are retained. Pixel equality does not establish semantic perfection. Cross-framework RGB and alpha are assessed separately against0.95 SSIM/28dB thresholds. Below are componentwise minima over the three distinct prompts in the first fresh native Omni process; the full per-prompt records identify potentially different worst samples.

Shape/task RGB min SSIM RGB min PSNR dB Alpha min SSIM Alpha min PSNR dB
512-t2i 0.991974273 38.658691 0.999529118 63.663978
512-edit 0.968473242 32.031133 0.999083354 60.656345
1024-t2i 0.972529273 29.234914 0.999028904 60.492056
1024-edit 0.997443726 51.741815 0.999331171 62.145452

Separate failure investigation

The complete original→current source diff is two files,16 insertions/12 deletions. Production changes rename the enable flags and pass one outer block flag by keyword instead of position. Arithmetic, kernels and weight storage are unchanged. The actual SM120 packing/restoration/LoRA regression passed at1024 and4096 tokens,2 tests/0 skips. This does not establish full-model LoRA or GPU compile performance.

A separately frozen original/current1024 native high run completed two fresh processes,44 PNGs and four Torch traces, all hash verified. Each task/source retains four qualification requests, two excluded warmups, four SYNC1 diagnostic requests and one separate profiled request. All44 images equal corresponding primary main RGBA outputs. Three complete interior profile steps (#2,#3,#4) per trace have identical GPU kernel sequence/launch geometry, CPU operator sequence/shapes and CUDA runtime counts:384 kernels,32 packed gate/up GEMMs and32 packed SiLU kernels per step,zero graph launches. Original/current sampled peaks are identical within this diagnostic (39339884544B T2I;45669089280B edit). This finds no operator/storage regression mechanism; Python argument-binding overhead is not isolated by these traces.

Task Source SYNC1 denoise p50 s (4 diagnostic requests) Profile GPU steps2/3/4 ms
t2i original 9.254464985 185.254570, 185.646476, 186.028302
t2i current 9.228657241 185.344291, 185.619332, 185.405699
edit original 10.718908526 210.659886, 210.604814, 211.152240
edit current 10.661360847 210.423547, 210.365308, 210.738846

These timings are diagnostic, one process per source, and cannot replace primary E2E or prove a performance repair. New resolution-specific main/current stage diagnostics and high→lossless→exact→high/BCG compatibility plans were not run after primary failure. No current-head BCG net benefit, new scenario or complete compatibility validation is claimed. No source patch is justified solely by matching traces.

Reproduction and archive

Use the pinned commits/checkpoint and an isolated owned GPU0, with the exact native launchers, contracts, warmups, process ordering and pair indices in evidence/plans/. run_qwen21_current_head_blocks1.py runs each AA or primary plan once; archive/verify/audit AA before primary. Preserve SYNC0 for primary. The separately named qwen21-cleanup-1024-forensics1 plan uses the unmodified generic diagnostic runner with SYNC1 and explicitly pins the failed primary receipt. Do not treat it as an admission retry. Local root-level remote scripts map to scripts/ and archived evidence/reproduction/ paths; remote root is /data/xyzhang/diffusion-rtx6000-20261026, mounted /task.

Complete archives remain local and are identified by public per-member hashes; public PNGs are deduplicated by content hash, complete raw numerical results/requests/telemetry are retained, response copies omit repeated base64 data with original hashes recorded, and traces are published separately from production code. All historical failures remain in the original PR evidence. The full frozen81-cell matrix is not claimed complete.

Complete evidence and numerical report · Archive hashes · Profile investigation

Actual fresh main/current outputs

These twelve panels are actual primary qualification outputs; full source pixels and alpha are preserved at 1:1. Labels state the failed cleanup status. The original historical media above remain unchanged.

Actual current-head t2i 512 00-image-short; cleanup status shown

Actual current-head t2i 512 01-image-long; cleanup status shown

Actual current-head t2i 512 02-image-text; cleanup status shown

Actual current-head edit 512 00-edit-short; cleanup status shown

Actual current-head edit 512 01-edit-long; cleanup status shown

Actual current-head edit 512 02-edit-chinese; cleanup status shown

Actual current-head t2i 1024 00-image-short; cleanup status shown

Actual current-head t2i 1024 01-image-long; cleanup status shown

Actual current-head t2i 1024 02-image-text; cleanup status shown

Actual current-head edit 1024 00-edit-short; cleanup status shown

Actual current-head edit 1024 01-edit-long; cleanup status shown

Actual current-head edit 1024 02-edit-chinese; cleanup status shown

Media provenance · Local visual inspection


CI States

Latest PR Test (Base): ❌ Run #42123750135
Latest PR Test (Extra): ✅ Run #42123749864
Latest PR Test (AMD ROCm 10): ⏳ Run #42123749967

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

@mintlify

mintlify Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit 44716a7.

Approve preview

@github-actions github-actions Bot added documentation Improvements or additions to documentation diffusion SGLang Diffusion labels Oct 28, 2026
@BBuf BBuf changed the title [Diffusion] Pack Qwen-Image 2.1 BF16 gate/up projections on SM120 [Perf] qwen-image21: Pack BF16 gate/up projections on SM120 Oct 29, 2026
@BBuf BBuf added run-ci CI: run the baseline test suite on this PR run-ci-nv-extra CI: NVIDIA/shared Extra suite (requires run-ci) labels Oct 29, 2026 — with ChatGPT Codex Connector
@BBuf
BBuf marked this pull request as ready for review October 29, 2026 03:23

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@BBuf

BBuf commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/kernels/ops/diffusion/test_model_fast_paths.py::test_qwen21_bf16_mlp_packing_restore_and_lora

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/kernels/ops/diffusion/test_model_fast_paths.py test_qwen21_bf16_mlp_packing_restore_and_lora

@BBuf

BBuf commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-failed-ci

1 similar comment
@BBuf

BBuf commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-failed-ci

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

diffusion SGLang Diffusion documentation Improvements or additions to documentation run-ci CI: run the baseline test suite on this PR run-ci-nv-extra CI: NVIDIA/shared Extra suite (requires run-ci)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant