Apply should_use_dp_reduce_scatterv guard to remaining MoE models (follow-up to #23731)

[ByronHsu]

Motivation

Follow-up to #23731 (Qwen3 MoE). Supersedes #23431 (same diff for the 12 files there) by also fixing hunyuan_v3.py.

PR #22642 introduced should_use_dp_reduce_scatterv(), which fuses the post-MoE all-reduce with dp_scatter into a single reduce_scatterv call inside LayerCommunicator. To avoid a double-reduce, the model-side tensor_model_parallel_all_reduce (or moe_*_all_reduce) on final_hidden_states must be skipped when this fast path is active.

That PR added the guard only to qwen2_moe.py. #23731 fixed qwen3_moe.py. Every other MoE model that does the same post-experts all-reduce silently double-reduces when running with DP attention + EP + moe_a2a_backend="none" — same regression pattern as #23729.

#23431 surfaced this in the nightly suite via test_minimax_m25.py variant TP8+DP8+EP8+DPAttn:

• 2026-04-13 (pre-Replace all-reduce + dp_scatter with reduce_scatterv for DP attention #22642): GSM8K 0.951 ✓
• 2026-04-16 → 2026-04-21 (post-Replace all-reduce + dp_scatter with reduce_scatterv for DP attention #22642): GSM8K 0.002 – 0.010 ✗
• After this guard: GSM8K 0.980 ✓ (matches the non-DPAttn baseline)

Modifications

13 files (~16 reduce sites). The 12 files from #23431 plus hunyuan_v3.py:

File	Reduce sites	Notes
bailing_moe.py	1	standard pattern
bailing_moe_linear.py	1	standard pattern
deepseek_v2.py	2	forward_normal + dual-stream; forward_cpu intentionally untouched (CPU path doesn't trigger the fast path)
exaone_moe.py	1	standard pattern
glm4_moe.py	2	forward_normal + dual-stream
hunyuan_v3.py	2	uses moe_expert_parallel_all_reduce + moe_tensor_model_parallel_all_reduce (same shape as qwen3_moe.py); both branches gated
llada2.py	1	standard pattern
llama4.py	1	standard pattern
mimo_v2_flash.py	1	standard pattern
minimax_m2.py	1	standard pattern
sarvam_moe.py	2	forward_normal + dual-stream
sdar_moe.py	1	standard pattern
step3p5.py	1	standard pattern

Each and not should_use_flashinfer_cutlass_moe_fp4_allgather() guard gets a sibling and not should_use_dp_reduce_scatterv() line, matching the pattern from qwen2_moe.py and qwen3_moe.py. hunyuan_v3.py does not have the fp4 guard, so a skip_post_reduce = should_use_dp_reduce_scatterv() local short-circuits both reduces.

Validation

• All 13 modified files import cleanly.
• Per-model GSM8K validation for minimax_m2 from Fix DP-Attention reduce_scatterv missing guard in MiniMax/Bailing MoE #23431: 0.060 → 0.980 on H200 4×GPU (TP=4 DP=4 EP=4) with all should_use_dp_reduce_scatterv() conditions satisfied.
• The patch only adds redundant-guard skip; numerics on every other path are unchanged.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results (see Fix DP-Attention reduce_scatterv missing guard in MiniMax/Bailing MoE #23431 GSM8K nightly).
• Follow the SGLang code style guidance.

cc @YAMY1234 (PR #22642 author)

Refs #23729 #23731 #23431

[gemini-code-assist]

Code Review

This pull request introduces the should_use_dp_reduce_scatterv() guard across several Mixture-of-Experts (MoE) model implementations, including Bailing, DeepSeek-V2, GLM-4, Hunyuan-V3, MIMO-V2, MiniMax-M2, Sarvam, SDAR, and Step3.5. This guard is integrated into the forward pass logic to conditionally skip the final tensor model parallel or expert parallel all-reduce operations when a fused reduction is expected to be handled by an external communicator. I have no feedback to provide.

[Kangyan-Zhou]

/tag-and-rerun-ci again