[LingYi-Liang]

Motivation

The contiguous SiTU path quantizes FP8 values using amax / 448, then rounds only the scale up to UE8M0 before the down projection. The GEMM therefore reconstructs those values with a different scale.

This affects SiTU activations when DEEPGEMM_SCALE_UE8M0 is enabled. It is separate from #43556, which fixes loading-time weight requantization.

Modifications

Round the scale to a power of two before quantizing, and pass that same scale to the existing packing helper.

Add two registered GPU regressions: one checks 42 input/shape combinations through the compiled packing helper; the other runs the real contiguous runner with FP8 and MXFP4 experts and two native DeepGEMM projections. Both retain the existing SiTU error bound of 17 * scale.

Accuracy Tests

Tested against main@8cd3c494a3aa3ae56dfd76f51bab6912234914dc on RTX 5070 Ti (SM120), WSL2, PyTorch 2.14.1+cu130, Triton 3.8.0 and sgl-deep-gemm 0.2.3.

• Both added test files fail assertions on the baseline, pass with the fix, and fail again when the old quantization calculation is restored.
• In the two-expert runner fixture, each weight format goes from 262,144 out-of-bound output elements to zero. The first GEMM is exact and the down projection is an identity, isolating activation quantization.
• A synthetic runner fixture using Kimi-K3's routed-expert dimensions (input/output width 3584, intermediate width 3072) also passes for both weight formats.

These are kernel and runner tests. No checkpoint, full-model accuracy evaluation, dispatcher integration or B200 run was performed. Targeted formatting, lint and registration checks passed; full pre-commit was not run.

Speed Tests and Profiling

A warm-kernel microbenchmark used CUDA events and six alternating baseline/fixed pairs. At 128 rows × 3072 output columns, the median of the per-pair medians increased from 2.599 µs to 2.856 µs; individual pairs were 8.3–20.3% slower. This is the measured cost of the corrected rounding path. The 8 × 1024 measurements were too noisy to attribute a change.

The measurement excludes packing, GEMMs, compilation and serving. End-to-end performance was not measured.

Checklist

• Format code with the full pre-commit workflow. Targeted checks passed.
• Add regression tests following the registered GPU test conventions.
• Update documentation. Not applicable: no public API or configuration change.
• Provide full-model accuracy and speed results. Not measured; component results are above.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Request review through Merge Oncall.
2. Obtain Codeowner and other required approvals.
3. Have an authorized user trigger the appropriate CI tests.
4. Merge only after the required checks and approvals.

---

CI States

Latest PR Test (Base): ❌ Run #39822592010
Latest PR Test (Extra): ❌ Run #39822592146
Latest PR Test (AMD ROCm 10): ❌ Run #39822592040