[ch-wan]

This PR is part of a merged stack (oldest at bottom):

• -> [Feat] spec: Support CUDA attention DP for DFlash and DFlash2 #46330 (merged)
• [Refactor] spec: Resolve draft parallel placement from model declarations #46290 (merged)

Motivation

CUDA currently rejects DFLASH with attention DP. Enable plain DFlash and selector-based DFlash2 using the draft placement resolved by #46290, including unequal local batches and idle DP replicas.

Modifications

• Admit CUDA attention DP with explicit --enable-dp-lm-head, CP=1 and PP=1. Reject Domino/LiLiCorr heads and the unimplemented experimental heterogeneous prefill/verify coordination mode.
• Reconstruct the embedding actually used by the draft on that embedding's own TP group before serving. Preserve padded/added vocabulary ordering, embedding quantization methods and output dtype.
• Defer DFLASH DP target-verify attention planning and graph-buffer loading to the runner, after final batch preparation and graph eligibility decisions. Stop marking deferred metadata ready through the deprecated skip_attn_backend_init argument.
• Initialize empty mRoPE positions for idle batches so DP padding can materialize valid target-verify rows.
• Read target DP state before entering the narrowed draft scope. Keep the draft eager under DP, retain eligible target CUDA graphs, and warm selector kernels during startup.
• Add focused unit tests, two registered four-GPU nightly E2E cases, and launch documentation.

The skip-argument changes are confined to DFLASH callers; EAGLE/NextN metadata decisions and the shared runner are unchanged. Empty idle mRoPE initialization is a shared ForwardBatch change and also applies to other algorithms using mRoPE models. Non-DP/DSpark preplanning defaults and NPU-specific behavior are preserved.

Accuracy Tests

Current head: 3a19e047cd4005fab35678d800a7d790f5bafd30, rebased onto main 6623ea445a287ff89fa61165a9d129daeee4ac34 after #46290 merged. Tests were not rerun for this landing rebase; the results below retain their original revision attribution. The parent now fixes the CPU source-audit failure by using sequence-checked DP slot reads in its draft-scope test. The DFLASH feature patch is unchanged by restacking.

Follow-up validation at f50467b918d3849db781dd5cd98e98fec4b96b9c: 180 CPU tests passed across the complete test_draft_tp_context.py (11) and test_runtime_context.py (169), including the previously failing full-checkout raw-slot scan. Changed-file pre-commit and whitespace checks passed. Only the inherited test changed from the previous stack head; production code and the architecture allowlist are unchanged. The original feature results below retain their original commit attribution; GPU tests were not repeated for this test-only fix.

Tested commit f31e240a083d25a5e6d21b5ea50f7f7be11c3bb8, based on #46290 at 42a50ed474230f2dc85652539307bbc304dd0dbd.

CPU: 100 passed, 1 skipped. Includes 9 new admission/embedding/metadata cases and 91 existing DFLASH, metadata-plan and fused-placement checks. The skipped Triton test requires CUDA. Changed-file pre-commit checks passed.

GPU: 4 × NVIDIA H200, TP4 / attention DP2. Deterministic FA3 target with FlashInfer draft, compared against target-only at the same TP/DP layout:

Target / draft	Greedy token IDs compared	Result
Qwen/Qwen3-4B / z-lab/Qwen3-4B-DFlash-b16	320	Exact match
Qwen/Qwen3.8-27B / incoai/Qwen3.8-27B-DFlash2	320	Exact match

Each fixture also passed a 64-token sampled smoke. Cases cover individual DP replicas, idle peers, concurrent unequal lengths and prefix reuse, with target CUDA graphs enabled. Additional FlashInfer target/draft eager and graph runs completed for both families; actual target graph execution was observed. A non-DP TP4 DFLASH graph smoke also completed.

Run the registered regressions with:

python test/registered/unit/spec/test_dflash_dp_attention.py
python test/registered/e2e/speculative/test_dflash_dp_attention.py

Environment: driver 595.71.05, Torch 2.14.1+cu130, FlashInfer 0.7.2, sglang-kernel 0.4.10, DeepEP 0.1.2.post1, DeepGEMM 0.2.3.post2 and Transformers 5.19.0.

Limits: exact parity is established for the deterministic, same-layout comparisons above; default FlashInfer target-only output itself varied with batch shape. This feature commit has no dedicated DP + EAGLE/MTP GPU regression. NPU, CP, PP, MoE transports, quantized embedding checkpoints, LoRA and multimodal inputs were not qualified on device. Documentation build checks were not run.

Speed Tests and Profiling

No performance benchmark was run; no speedup claim.

Checklist

• Changed-file pre-commit checks pass.
• Add and run focused unit and GPU tests.
• Document supported layouts and launch flags.
• Report accuracy results and validation limits.
• Follow SGLang code style.

---

CI States

Latest PR Test (Base): 🚫 Run #40162290440
Latest PR Test (Extra): 🚫 Run #40162290449
Latest PR Test (AMD ROCm 10): ⏳ Run #40162290559

[ch-wan]

/rerun-test test_dflash_dp_attention.py‎

[github-actions]

⛔ test_dflash_dp_attention.py: Ambiguous filename test_dflash_dp_attention.py — matched 2 files:

• test/registered/e2e/speculative/test_dflash_dp_attention.py
• test/registered/unit/spec/test_dflash_dp_attention.py

Please provide the full path, e.g. /rerun-test test/registered/unit/spec/test_dflash_dp_attention.py

[ch-wan]

/rerun-test ‎test/registered/e2e/speculative/test_dflash_dp_attention.py

[github-actions]

🚀 4-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/e2e/speculative/test_dflash_dp_attention.py

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
sglang-doc	🟢 Ready	View Preview	Oct 25, 2026, 4:37 AM

💡 Tip: Enable Automations to automatically generate PRs for you.