[unclezhou486]

Motivation

DSpark speculative decoding with a DeepSeek-V4 (MoE) draft supports DP attention
(attn_tp_size < tp_size) on Ascend NPU, but the draft attention's output
projection wo_b reduced its partial results over the wrong process group,
which corrupted the draft output and collapsed the accept rate to ~0.

DSparkAttention (python/sglang/srt/models/deepseek_v4_dspark.py) subclasses
MqaAttentionBase and hard-codes wo_b_reduce_results=True, overriding the
base's default reduce_results = (attn_tp_size == tp_size and attn_tp_size > 1).
As a result, under DP attention (attn_tp_size < tp_size) wo_b still performed
a row-parallel all-reduce, but over the full DP × TP group, while its weight
is sharded over the attn-TP group only. That summed partial outputs that
belong to different DP ranks' different batches, so the draft attention output
was garbage and the accept rate collapsed to ~0.

A guard in DSparkWorkerV2 rejected this whole configuration
(attn_tp > 1 + DP attention + MoE draft), which blocked the feature on NPU.

(Note: the guard's error string referred to the "MoE-under-DP all-reduce", but
the actual corruption was the draft attention's wo_b reduce group; the draft
MoE path itself handles attn_tp > 1 via _dp_gather_via_all_gather.)

Modifications

1. python/sglang/srt/models/deepseek_v4.py

   • MqaAttentionBase.wo_b now passes
     use_dp_attention_reduce=is_dp_attention_enabled(), so its row-parallel
     all-reduce runs over the attn-TP group (get_parallel().attn_tp_group)
     when DP attention is enabled, matching the weight sharding.
   • DSparkAttention.forward calls self.wo_b(...) directly (it does not use
     _project_wo_b), so this single flag fixes the draft reduce with no
     double-reduce.

2. python/sglang/srt/speculative/dspark_components/dspark_worker_v2.py

   • Allow the attn_tp > 1 MoE-draft + DP-attention configuration on NPU by
     adding and not _is_npu to the guard. CUDA/ROCm still reject it.

3. python/sglang/srt/hardware_backend/npu/attention/ascend_dsv4_backend.py

   • Pad the draft ori_sparse_indices tail with -1 ("skip") up to the
     attn-TP-padded token count, matching out_cache_loc
     (prepare_mlp_sync_batch pads the query to an attn_tp multiple, but
     seq_lens is not).

Accuracy Tests

Model	Dataset	Metric	Subset	Num	Score
DeepSeek-V4-Pro-0813-w4a8	GPQA-Diamond	Accuracy ↑	gpqa_diamond	198	88.9%

Speed Tests and Profiling

Model	Dataset	Num	Avg Lat	Avg TTFT	Avg TPOT	Avg Thpt	Avg In	Avg Out
DeepSeek-V4-Pro-0813-w4a8	GPQA-Diamond	198	227.141 s	8876.8 ms	44.7 ms	25.74 tok/s	252	5847

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ✅ Run #39185582451
Latest PR Test (Extra): ❌ Run #39185582349
Latest PR Test (AMD ROCm 10): ❌ Run #39185582366

[sglang-npu-bot]

/tag-run-ci-label

[unclezhou486]

/rerun-failed-ci

[unclezhou486]

/rerun-failed-ci try again

[unclezhou486]

/rerun-failed-ci try again again

[unclezhou486]

/rerun-failed-ci try again again again

[unclezhou486]

/rerun-failed-ci try again again again again again again again again again again again again again
again again

[unclezhou486]

/rerun-failed-ci again again again again again again again again

[unclezhou486]

/rerun-failed-ci again again again again

[unclezhou486]

/rerun-failed-ci again
again again again again again again again again