Guard hc_split_sinkhorn against DP attention's empty idle batch

[zijiexia]

Motivation

Targets the dsv4.1 branch (#38798) rather than main, since that is where DeepSeek-V4.1
lives today.

Serving deepseek-ai/DeepSeek-V4.1-Flash with --enable-dp-attention on H200 kills every
idle rank during the startup warmup, so the server never becomes ready:

tvm.error.InternalError: Check failed: (wl.grid_dim(0) > 0 && wl.grid_dim(1) > 0 && wl.grid_dim(2) > 0)
  is false: CUDALaunch Error: grid dimension must be positive, but got grid=(0,1,1)
  File "<unknown>", line 0, in __tvm_ffi_hc_split_sinkhorn_kernel_

with the Python frames ending in the DP-attention idle path:

  sglang/srt/model_executor/runner/eager_runner.py, line 225, in execute
  sglang/srt/model_executor/runner/eager_runner.py, line 459, in _execute_idle

Precisely the ranks with no work die — 7 of 8 at --tp 8 --dp-size 8, and 6 processes at
--tp 8 --dp-size 4 (3 idle DP groups x 2 TP ranks) — i.e. every rank that is idle while the
single warmup request is served.

hc_split_sinkhorn derives its grid from the token count and has no empty-batch early-out,
so a zero-token forward reaches the TileLang kernel and launches a zero-sized grid. Both call
sites in models/deepseek_v4.py (the two mHC pre-norm paths) call it unconditionally.

Ruled out with single-variable runs on the failing host, so the report is not confounded:

variable	result
--cuda-graph-max-bs-decode 64	not it — GB300 works with and without it
--attention-backend dsv4 / --moe-runner-backend flashinfer_mxfp4	not it — H200 fails with them dropped
dp_size 8 vs 4	not it — H200 fails at both

4x GB300 at --tp 4 --ep-size 4 --dp-size 4 --enable-dp-attention starts and serves fine, so
in practice this bites Hopper: Blackwell appears never to reach the kernel with zero tokens
(its idle batch stays inside a captured CUDA graph) while Hopper takes the eager idle path.
I did not instrument GB300 to prove that, so I would not state it as more than an
observation — the missing guard itself is unconditional, and the fix is not arch-specific.

Modifications

One early-out in hc_split_sinkhorn, returning correctly shaped empty tensors. It sits ahead
of the is_gfx1250_supported() branch so the Triton port is covered too, and for a non-empty
batch it is a no-op.

Accuracy Tests

Reproduction and fix both on 8x H200, real deepseek-ai/DeepSeek-V4.1-Flash weights, this
branch's tree:

sglang serve --trust-remote-code --model-path deepseek-ai/DeepSeek-V4.1-Flash \
  --tp 8 --ep-size 8 --dp-size 8 --enable-dp-attention \
  --mem-fraction-static 0.8 --attention-backend dsv4 --moe-runner-backend flashinfer_mxfp4 \
  --max-running-requests 256 --cuda-graph-max-bs-decode 64 --host 127.0.0.1 --port 30000

check	before	after
server reaches ready	no — 7/8 ranks die in warmup	yes
/server_info	—	dp_size=8, enable_dp_attention=true, tp_size=8
4 sequential requests (7 idle ranks each)	—	4/4 correct (15% of 240 is **36**)
16 concurrent requests	—	16/16 correct
server still up afterwards	—	yes
non-DP run, same patched tree	ok	ok — guard is a no-op for non-empty batches
4x GB300 --tp 4 --dp-size 4	already worked	still works

No second zero-grid kernel surfaced behind this one; the sinkhorn launch was the only
unguarded site on the mHC idle path.

Environment: H200 (CC 9.0), PyTorch 2.13.0+cu130, sglang-kernel 0.4.6.post1,
flashinfer 0.6.18, triton 3.7.1. Counter-example host: 4x GB300 (CC 10.3), same tree.

Speed Tests and Profiling

Not applicable — the guard only runs for a zero-token batch, which previously crashed.

Checklist

• Format your code according to the Format code with pre-commit.
• Provide accuracy benchmark results according to Test the accuracy.

Note

The published lmsysorg/sglang:dev-dsv41 image predates #38804 and still carries the older
guard "V4.1 vision currently supports TP/EP without DP, CP, PP or MoE A2A", so it rejects
--enable-dp-attention outright — this crash is only reachable on a build that includes that
commit, i.e. this branch.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ❌ Run #34536638902
Latest PR Test (Extra): ❌ Run #34536638839
Latest PR Test (AMD ROCm 10): ❌ Run #34536638987