[DSv4] Loading Time Weight Dequant

[laixinn]

Motivation

Current available weights for DSv4 Flash (from deepseek-ai or sgl-project) do not support TP8, which performs better in H20.
Dequanting FP4 to FP8 would be more preferrable during weight loading. Because this process depends on the TP size.
usage:

SGLANG_DSV4_FP4_DEQUANT=1 \
python -m sglang.launch_server \
  --model deepseek-ai/DeepSeek-V4-Flash/ \
  --tp 8 \
  --tool-call-parser deepseekv4 \
  --reasoning-parser deepseek-v4 \

cc @AniZpZ #23602

Accuracy Tests

MMLU tests pass.

Benchmark

Setup:
1P1D 8xH20, radix cache off
Prefill: TP8+CP8
Decode: TP8
ISL: 15k, OSL: 700

Results:
batch size = 1

Metric	TP8	TP4	Speedup
P50 TPOT (ms)	484	753	1.56x
P50 TTFT (ms)	3.13	3.43	1.10x

qps=0.83

Metric	TP8	TP4	Speedup
P50 TPOT (ms)	699	1557	2.23x
P50 TTFT (ms)	3.98	4.43	1.11x

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ✅ Run #28561076492
Latest PR Test (Extra): ❌ Run #28561076381

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[laixinn]

/tag-and-rerun-ci

[AniZpZ]

/tag-and-rerun-ci

[AniZpZ]

LGTM

[AniZpZ]

@b8zhong hi, could you please review this pr，the ci passed

[BBuf]

In ModelConfig, the new logic uses envs.SGLANG_DSV4_FP4_DEQUANT.is_set() to decide whether to enter the dequant path. is_set() only checks whether the env var exists, so explicitly setting SGLANG_DSV4_FP4_DEQUANT=0 or false is still treated as enabling the feature. Then set(self.is_fp4_experts is not None) writes True because self.is_fp4_experts is always a bool here.

Could we use .get() for the enable check, and write back self.is_fp4_experts instead?

[laixinn]

Done.

[BBuf]

When is_fp4_experts and dequant_fp4_to_fp8 is true, Fp8Config.get_quant_method() returns the plain Fp8MoEMethod. That works for the normal FP8-compatible backend path, but if the user also passes --moe-runner-backend flashinfer_mxfp4, Fp8MoEMethod.create_moe_runner() does not create self.runner for that backend. Later apply() can hit an unsupported/missing-runner path.

Could we either reject this combination with a clear error, or normalize the backend to an FP8-compatible backend when dequant is enabled?

[laixinn]

Done.

[laixinn]

@BBuf Comments solved. please check.

[ispobock]

/rerun-failed-ci