[BBuf]

Motivation

This is the second pass of #44690. Three weeks after that PR, python/sglang/kernels had gained another 66K lines, python/sglang/multimodal_gen 43K and test/registered 162K. This PR removes what in that code is dead or redundant. More importantly, it fixes the places where CI is green without testing anything, and the runtime bugs found along the way.

433 files, +5,551 / −13,052.

• About 1.4K of the additions are tests moved to test/manual/, because no CI runner has the hardware or package they need.
• About 0.8K are shared test helpers that replace copies.

Bugs fixed (please review these first)

Area	Problem	Fix
Kernel CI	base-b-kernel-unit-test-8-gpu-h200 has no registered test. On every kernel PR and three times a day it books an 8×H200 runner, installs everything, and logs No tests found.	Matrix row and suite removed.
Kernel CI	Tests registered on lanes whose hardware never satisfies their skip gate. The production kernels behind them had no CI at all.	Registered on hardware that runs them, and each was run there: • SM120 FlashMLA fallback (Ran 15; skipped=15 on H100) • Blackwell FP4 indexer quant (36 of 39 cases skipped) • Q8KV8 sparse prefill (25 cases) • FlashInfer KDA prefill graph and input pack • SM100 Mamba stochastic rounding • the FA3 extend-init regression • KDA MTP slot stride • Gemma4 Blackwell sampling split • gfx950-only kernels on the MI300 lane • diffusion FA4 forward_into • SubBlock sm100/sm120 kernels (one case skipped on sm_100, see below) • 4-rank and 2-rank diffusion cases on 1-GPU lanes
Q8KV8 sparse prefill	Once the Blackwell cases ran, 15 of them failed on main. #45909 packed each request's compressed history contiguously and updated the H100 cases, but nobody ran the Blackwell cases, so their expected offsets stayed stale. Their output-vs-paged-attention assertion passes.	Registration and expectation fix land together; 37 passed on sm_103.
Kernel tests	test_kpool_topk_transform_fused.py runs under unittest.main(), which drops the module-level pytest functions. 4 of its cases (ties, overfull bins, page-table mapping under graph replay) never ran.	pytest.main entry; scanned the kernel tests for the same pattern.
Kernel tests	Assertions that cannot fail: • a tensor compared with itself (test_rmsnorm); • an AMD all-reduce "determinism" test with no assert, which only prints ✓/✗ and tests RCCL rather than SGLang; • *_jit_vs_aot comparisons against AOT ops that do not exist (always skipped); • a MoE wna16 GEMM test that always skipped (it now checks against a dequant reference and runs its 24 cases).	Fixed, or deleted where nothing SGLang-owned was tested.
Unit tests	Triton kernels tested only from CPU-registered files, so their CUDA cases never ran: row_logsumexp[_topk], free_unbind_inplace, gather_page64_kv_latent, the token-id translation Triton path.	GPU registrations added (5090).
Unit tests	Tests of copies of the code instead of the code: • the decode CUDA-graph profile options (a test-local copy); • four MoE shared-expert fusion gates asserting only assertTrue(reason), which the CPU early-return satisfies; • test_h3_request.py, which failed 8 of 12 cases when run alone because its ComfyUI stubs had rotted. It passed only if another test happened to import the real module first.	The tests now call the real method and assert each branch's message. The stubs are fixed.
CI registration	test_gated_launch.py used nightly=True on a CUDA registration, so no job ever ran it.	Flag removed; it now runs in nightly. check_registered_tests.py rejects nightly=True on CUDA/CPU.
CI tooling	update_est_time.py matched registrations with a single-line regex. It missed 150 of the 2,459 CUDA/CPU registrations (wrapped, positional, float), so their est_time never refreshed.	The literal is found through the AST: 0 missed.
Diffusion	--srt-encoder-connection-timeout was stored under a name that is not a field, so the documented flag (and the one GLM-Image's timeout error recommends) was silently dropped. type=int also rejected the documented default 3.05.	dest= the field, type=float.
Diffusion VMoBA	The runtime imported kernel.attn.vmoba_attn.vmoba, which exists nowhere. The chunk-selection fix in #44286 went into the in-tree copy that nothing imported, and its tests sat outside CI discovery.	The runtime and resolver import the in-tree module. Its tests moved into multimodal_gen/test/unit.
Diffusion exact tier	Wan and LingBot fuse Q/K/V into one GEMM unconditionally. At the model shapes it is bit-identical on H200 and GB300. At M ≤ 256, cuBLAS picks split-K and the result differs, so exact requests were not bit-identical. Wan's fused RMSNorm+RoPE had no check either.	The QKV fusion is gated at lossless (same math, rounding moves), the default tier, so default behavior is unchanged. The norm+RoPE fusion goes through BitExactFusionGate.run.
MiniMax-H3	The pipelined Ulysses attention never ran its first-call equality check. Enabling the check exposed that the sequential reference left non-zero pad rows that #47566 zeroes, which would have turned the pipeline off.	The reference zeroes the alignment rows (never read downstream); the check runs and passes (same=1).
Registry	gemm.dsv3_fused_a_gemm pointed at sgl_kernel:dsv3_fused_a_gemm, which does not exist. The namespace test only checked that the module existed.	Spec removed. test_inventory_targets_resolve now resolves every target's attribute statically (incl. the AOT package) on the CPU lane.
CuTe DSL GDN/KDA decode	The wrappers accepted scale, use_qk_l2norm_in_kernel, softplus_beta and softplus_threshold, then compiled with fixed values. Today's callers pass exactly those values.	Non-matching values raise ValueError (+ test).
ROCm block-FP8	use_w8a8_block_fp8_matmul_unrolledx4 evaluated its condition without return, so production never selected the unrolledx4 kernel. Only the tuner ran it.	Kernel, selector and tuner branch removed. Runtime behavior is unchanged.
Unlimited-OCR / QK-norm gate	Their PDL probes checked capability >= 9, which is true on gfx942/950, so they emitted PDL on ROCm.	current_arch().supports_pdl.

Modifications

Kernels (python/sglang/kernels, 108 files, −3.1K net)

• Dead code (each re-checked by whole-repo grep, incl. string targets):
  • 865 lines of jit/csrc/moe/expert_specialization/*.cuh that nothing loads;
  • ngram decode fast paths that were never wired in, and their benchmarks;
  • the MuseSpark router_gemm JIT kernel (the model only uses its token limit);
  • the test-only TextSeal standalone selector;
  • two unused LoRA MoE custom ops;
  • LPLB shmem helpers, row_argmax and create_argmax_workspace;
  • the f32x2 helpers left by a reverted PR;
  • finish_flat_indexer_topk;
  • the small_field_copy descriptor path;
  • an unloadable LoRA plans table;
  • losing mla_qk_prep variants (the default path is byte-identical on H200 and GB300 over 64 cases);
  • unused use_tma/topk/config parameters;
  • the MuseSpark softmax routing branches;
  • aliases, constants and re-exports nothing uses.
• Classification ([Refactor] kernels: Regroup operators by computation and dissolve ops/muse_spark #45112 rules):
  • engram_gather_peer (peer-memory read) → communication.
  • MSA-v1 compact_dequant (KV-cache gather + dequant) → kvcache.
  • The two diffusion op ids that targeted quantization/activation modules are renamed to their real groups; their facade re-exports are gone.
  • qwen_moe_topk_output moves to its only caller in qwen2_moe.py.
  • The gfx1250 per-token FP8 quant → quantization.
• Arch queries: about 15 sites that still probed the device by hand now use sglang.kernels.arch ([Refactor] kernels: Centralize GPU architecture queries in sglang.kernels.arch #47782) with the same semantics. argmax/softmax JIT adopt max_vec_bytes. Import-time device-0 probes moved to call time.
• Dedups:
  • the gfx950 gluon MoE helpers (identical AMDGCN for all 68 cases);
  • the hipcc compile-and-load block;
  • the SM90 FP8 CuTeDSL LoRA class, now a subclass of its BF16 sibling (bit-exact on H200);
  • the gfx950 vattn HIP binding.
• Env: raw os.environ reads in kernels/models are declared in environ.py with the same names and defaults. Two settings are removed:
  • SGLANG_HIP_MHC_BOUNDARY_WIDE_TILE, a kill switch for a bitwise-identical tile;
  • SGLANG_XPU_LN_MOD_NUM_WARPS, a layout-only setting.
• HiCache: the page-unified layout is admitted only if both the write-back and the load-back JIT probes pass; until now the load-back probe was never called.
• Docs: cookbook lines that told users to set env vars nothing reads are removed.

Diffusion (python/sglang/multimodal_gen, 118 files, −1.5K net)

• Dead code:
  • 7 disaggregation flags nothing read;
  • --mask-strategy-file-path (sliding-tile attention reads --attention-backend-config mask_strategy_file_path=..., and the error now says so);
  • the /stats timeout counter that was always 0;
  • vestigial PortArgs fields that leaked three temp files per construction;
  • about 25 unused helpers, constants and config fields;
  • test oracles moved out of runtime modules into the tests.
• Shared helpers:
  • four more hand-written verify-once gates use BitExactFusionGate.run / can_attempt (flux3 video VAE, FLUX.2 XPU, KL-VAE, Qwen-Image-2.1);
  • Wan and LingBot share one row-packing helper;
  • the duplicated Nomas helpers, OpenAI image/video/mesh API helpers and flux3's copy of the launcher conditions are merged;
  • the two disagreeing "unrecoverable device error" classifiers are now one.
• Docs:
  • --enable-attention-backend-autotune was documented but not a CLI flag; it is now one.
  • ring_sp_performance.mdx documented a flag that does not exist; it now uses the real --kv-gather-degree.

Tests and CI

• Deleted:
  • tautologies (argparse pass-through, default and constant echoes, inspect.getsource/AST source tests, mock-the-SUT, torch-math);
  • duplicate cases and files;
  • benchmarks of test-local copies;
  • always-skipped tests;
  • #47603 leftovers (dead model lists, a test that returns in CI);
  • tests with a hard-coded developer path.
• Merged copy-pasted fakes, fixtures and oracles into python/sglang/test/ helpers. Two test classes were collected twice through imports (45 extra runs per CPU job).
• Trimmed grids with no coverage gain: page-unified dtype sweep (252 → 168 cases), MiniMax decode top-k (144 → 48).
• Faster CPU gate:
  • test_runtime_context.py: 132 s → 13 s;
  • test_flashinfer_autotune_sync.py: 30 s → 10 s;
  • three one-purpose files merged.
• Moved to test/manual/: tests that need NCCL ≥ 2.31.2, a private checkpoint, an unset env var, or 4 GPUs of a manual-only class; plus a benchmark CLI that lived in a registered test.
• CI:
  • the empty 8-GPU XPU nightly job is removed;
  • stale suite names, globs and rerun_test_groups entries are removed;
  • the Muse Spark HF token follows the runner, not the stage;
  • comfy-kitchen (no deps) is installed for the diffusion unit/B200 jobs, so its three tests stop skipping.

Behavior changes

• Diffusion CLI:
  • the 7 unused disaggregation flags and --mask-strategy-file-path are now rejected;
  • --srt-encoder-connection-timeout takes effect and accepts floats.
• Diffusion exact requests no longer use the fused QKV GEMM (Wan, LingBot). lossless and high outputs are byte-identical to main.
• HiDream-O1 rejects --flow-shift/--scheduler-class-override, as it already does for its other fixed settings. HunyuanImage-3 honors a user --flow-shift instead of overwriting it.
• /v1/images/edits returns 400 when mask is given; it was silently ignored.
• The diffusion client treats CUDA error:, HIP error: and torch.AcceleratorError as unrecoverable, like the worker does.
• ROCm: no PDL in the Unlimited-OCR DASH / QK-norm gate kernels. The gfx1250 kpool indexer uses Triton act_quant, like the other two DSA paths.
• CuTe DSL LoRA prepare: SM120 now gets a clear NotImplementedError instead of compiling the SM100 class and failing.
• Removed env vars: SGLANG_HIP_MHC_BOUNDARY_WIDE_TILE, SGLANG_XPU_LN_MOD_NUM_WARPS. Newly declared boolean envs (SGLANG_ROCM_SMALLM_FP8_BS, SGLANG_DISABLE_LFM_FUSED_CONV) parse like every other EnvBool.

Left alone (owner decisions, not in this PR)

• AMD/NPU suites no workflow runs: 55 AMD files (~8.9K lines) and 8 NPU files (~1.5K lines) register suites that no workflow invokes. run_suite.py does not validate AMD/NPU suite names. The list is in a comment below.
• Cake softmax on SM100: since the JIT branch went first ([Perf] sampling: JIT softmax and argmax for the sampler and speculative verify #47512), the Cake softmax can no longer be selected; its input domain is inside the JIT one.
• MiniMax-H3 4-GPU H100 smoke suite: FILE_SUITES["4-gpu-h100"] has no bridge file and no workflow, so its three cases never run.
• Code waiting on [Feat] [AMD] minimax: Support MiniMax-M3.1-Flash on MI350X / MI355X (gfx950) #46693: the gfx950 kernels from [Feat] [AMD] minimax: gfx950 sparse attention, NVFP4 KV and DSpark verify kernels #46690–[Feat] [AMD] distributed: JIT quick all-reduce for TP4 on gfx950 #46692 that [Feat] [AMD] minimax: Support MiniMax-M3.1-Flash on MI350X / MI355X (gfx950) #46693 wires.
• Auto-residency probe: unreachable since [Perf] diffusion: Skip the residency calibration probe in auto mode until a planner consumes it #45979 ("until a planner consumes it").
• SubBlock ragged tail on sm_100: TestSubBlockNumerics::test_ragged_tail_reproduces_dense returns NaN on the B200 lane and passes on sm_103 and sm_120. The kernel is not changed here, so the case is skipped on sm_100 until the tail path is fixed.
• topk_sigmoid on ROCm: every case of test_moe_topk_sigmoid.py failed on the MI300 jit-kernel lane, so the test stays CUDA-only.
• pr-test-sgl-kernel.yml benchmark loop: || echo means the job cannot fail.

Accuracy Tests

Full regression sweep, base (4c7c775c) vs this branch, same box. Each test file runs in its own pytest process. Failing test ids are compared:

Box	Files	Test ids failing only on this branch
H200 (sm_90)	test/registered/kernels/ops (448), test/registered/unit (1153), multimodal_gen/test/unit (395)	0
GB300 (sm_103)	test/registered/kernels/ops (448)	0 (15 fewer: the Q8KV8 Blackwell cases)
RTX 5090 (sm_120)	test/registered/kernels/ops (448)	0 (the same 833 pre-existing ids)

The remaining failures are identical on both trees: multi-GPU tests on a 1-GPU box, optional packages missing from the image, and kernels that need hardware the box lacks.

End to end, fixed seed, base vs this branch. An instrumented run confirmed the fused paths were active on both sides.

Model	Box	Result
Wan2.1-1.3B	H200	lossless, exact and high mp4 byte-identical; fused QKV GEMM active (M=14040); norm+RoPE gate verified
Wan2.1-14B	GB300	lossless and exact byte-identical
Qwen-Image-2.1	H200	byte-identical; 10 DiT gates verified (incl. the converted ones)
LingBot-World realtime (CI case)	H200	exact and lossless raw frames identical; fused GEMM active under lossless
MiniMax-H3 T2VA, Ulysses 2	GB300	lossless and exact byte-identical; pipelined attention first-call check same=1

Kernel dedups, bit-for-bit (torch.equal on dumped outputs, base vs branch):

• the CuTeDSL FP8 LoRA class merge, gelu_and_mul_triton, MuseSpark routing, argmax and the SM10x JIT softmax: 88 tensors on H200 and 112 on GB300;
• mla_qk_prep: 64 cases on H200 and GB300;
• the gfx950 gluon helper hoist: identical AMDGCN for all 68 cases (372 compiles via Triton's AMD backend).

Lane changes were run on the hardware of their new lane:

• SM120 cases on an RTX 5090;
• SM100/SM103 cases on GB300;
• H100-lane cases on H200;
• the new 4-GPU diffusion cases on 4×GB300.

CI tooling:

• pre-commit run --from-ref 4c7c775c passes over the whole range, including the workflow checks.
• check_registered_tests.py passes; on main it fails, naming the nightly=True file.
• Suite selection was simulated with run_suite.filter_tests, and compute_partitions.py gives the same shard counts for pr-test, pr-test-extra, nightly and weekly.
• update_est_time.py dry run: 1402 updates, up from 1372; the old 1372 keep their values.
• The diffusion partition planner reports 0 missing estimates, and both newly scheduled 2-GPU files ran on 2×GB300.
• An import walk over sglang.{kernels,multimodal_gen,srt,test} finds no new import failures.
• After merging main 851642ad, the 18 test files touched by the newly merged upstream commits pass on main and on this branch.

Not validated:

• true SM100 / B200 (no B200 machine was available; sm_103 stood in for every major == 10 gate);
• ROCm, NPU, XPU and MUSA (those changes are mechanical and import-checked);
• the VMoBA CUDA path (FlashAttention 2 is not in the image);
• FLUX.3 e2e (gated weights);
• HiDream-O1, HunyuanImage-3 and Nomas e2e (unit tests only).

Speed Tests and Profiling

No perf-path changes intended:

• Default-tier fused paths are unchanged.
• The removed kernels and branches were not selected in production.
• The mla_qk_prep default variant is byte-identical.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ⏳ Run #42123536075
Latest PR Test (Extra): ❌ Run #42123536315
Latest PR Test (AMD ROCm 10): ⏳ Run #42123536434

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

[mintlify]

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit 41ac5ac.

Approve preview

[mickqian]

/tag-and-rerun-ci

[BBuf]

/tag-and-rerun-ci