feat(deepep_v2): validate Qwen3.5 MoE for --moe-a2a-backend deepep_v2

[whn09]

Motivation

--moe-a2a-backend deepep_v2 is rejected outright on every Qwen3.5 MoE checkpoint, because
validate_deepep_v2_model_architecture does not list the architecture:

ValueError: DeepEP v2 MoE is not validated for 'Qwen3_5MoeForConditionalGeneration';
  supported architectures are ['DeepseekV3ForCausalLM', 'DeepseekV4ForCausalLM',
  'Qwen3MoeForCausalLM']. Other model workflows may require an all-reduce after A2A
  combine. Use --moe-a2a-backend deepep.

Unlike #39080, no model-side change is needed here. Qwen3.5's MoE layer is
Qwen2MoeSparseMoeBlock (models/qwen3_5.py:952 and :1204), and that block is already
v2-aware on main @ 66c7bc8. The gate's own comment states the two conditions an
architecture has to satisfy — "take the A2A MoE path and skip post-expert all-reduce" —
and both hold without touching the model file:

condition	where it is already satisfied (main @ 66c7bc8)
takes the A2A MoE path	qwen2_moe.py:747-751 — forward() returns _forward_deepep() when is_deepep_v2()
…with EP geometry, not TP	qwen2_moe.py:397-401 — the same predicate drives the ep_size / num_experts / top_k block
skips post-expert all-reduce	qwen2_moe.py:811-818 — the final all-reduce is guarded by should_skip_post_experts_all_reduce(is_tp_path=True)
shared-expert 1/ep_size pre-scale not double-applied	qwen2_moe.py:443 and :459 — is_deepep_class_backend() already covers v2

So this PR is a whitelist entry and nothing else.

Modifications

arg_groups/moe_hook.py: "Qwen3_5MoeForConditionalGeneration" added to
validate_deepep_v2_model_architecture's validated_architectures, after
"Qwen3MoeForCausalLM".

Deliberately not claiming the text-only Qwen3_5MoeForCausalLM (qwen3_5.py:1962, plus
the wrapper at qwen3_5_text.py:223). It uses the same Qwen2MoeSparseMoeBlock and would
very likely work, but I have only run the ConditionalGeneration checkpoint, and this
whitelist means "someone has actually run it". Same for
InternS2PreviewForConditionalGeneration, which subclasses the Qwen3.5 model but registers
its own architecture string.

Accuracy Tests

Numerics against the DeepEP v1 path (--moe-a2a-backend deepep) on the same weights and
prompts. v1 is the right control: it is the established EP dispatcher, so a difference is
attributable to the dispatcher rather than to EP-vs-not. Both arms at matched
TP=EP=8, matched --chunked-prefill-size 4096, matched --moe-runner-backend deep_gemm,
so the dispatcher is the only variable.

Qwen/Qwen3.5-35B-A3B-FP8 (256 experts, top-8, hidden_size=2048, 128×128 blockwise FP8),
1 node × 8 B300, greedy, first-generated-token top-5 logprobs over a fixed 12-prompt set:

	result
top-1 token	0 / 12 mismatches
top-5 set / order	0 differences
bit-identical logprobs	10 / 12 prompts
worst |Δlogprob|	1.598e-01, on the two prompts longer than 800 tokens

The two differing prompts are inside this path's own run-to-run noise, and it is worth
measuring that noise rather than assuming a threshold. Capturing the same v1 server
twice reproduces the same pattern with larger deltas on the same two prompts:

prompt	v1 vs v2	v1 vs v1 (same server, 2nd capture)
3504 tok	6.600e-02	1.761e-01
809 tok	1.598e-01	3.161e-01
other 10	0.000e+00	0.000e+00

That control also produced a top-5 order change that the v1-vs-v2 comparison did not. The
mechanism is in-tree: both scatter kernels claim each expert's destination slots with
tl.atomic_add (kernels/ops/moe/ep_moe_kernels.py:1120), so token order within an expert
varies per run and DeepGEMM's accumulation order follows. sglang says as much itself —
deepep_v2 refuses --enable-deterministic-inference on the grounds that it "does not
forward deterministic=True to ElasticBuffer, so deterministic sorting remains disabled"
(moe_hook.py:328-334). Short prompts have little per-expert contention and stay
bit-reproducible; long ones do not. A fixed absolute threshold applied to this comparison
measures that, not the dispatcher.

Per-prompt criterion used instead: |Δ|(v1, v2) <= max(1e-3, |Δ|(v1, v1')). 12/12 pass,
with 0 top-1 mismatches.

Sanity check that the flag is actually doing something after the patch — this line appears on
all 8 ranks, and does not appear at all on unpatched main (the server never gets that far):

Initialized DeepEP v2 ElasticBuffer: world_size=8 hidden_size=2048 num_topk=8
  max_dispatch_tokens_per_rank=4096 use_fp8_dispatch=True allow_hybrid_mode=True

Reproducing

deepep_v2 requires a 128×128 blockwise FP8 checkpoint with dynamic activation scaling
(_validate_deepep_v2_quant_method, fused_moe_triton/layer.py:242), and there is no
load-time conversion path, so a BF16 Qwen3.5 checkpoint is rejected with
weight_block_size=None and --quantization fp8 gives per-tensor scales. Use the FP8
checkpoint:

SGLANG_DEEPEP_V2_NUM_MAX_DISPATCH_TOKENS_PER_RANK=4096 \
python3 -m sglang.launch_server \
  --model-path Qwen/Qwen3.5-35B-A3B-FP8 \
  --tp-size 8 --ep-size 8 \
  --moe-a2a-backend deepep_v2 --deepep-v2-mode hybrid \
  --moe-runner-backend deep_gemm --attention-backend trtllm_mha \
  --chunked-prefill-size 4096 --cuda-graph-max-bs 64 \
  --mem-fraction-static 0.7 --trust-remote-code

The env var matters: its default of 128 is below the prefill budget, and the launch fails with
required=16384, capacity=128 before reaching anything model-specific. --deepep-v2-mode hybrid is required on this fabric (EFA); direct keeps ginStrongSignalsRequired=true,
which EFA's GIN provider does not implement.

Speed Tests and Profiling

Not included. I have a single 8-GPU node, so both arms would run entirely over NVLink and the
resulting throughput numbers would say nothing about the a2a path at the scale deepep_v2
targets — I would rather not present a number that reads as a benchmark of it. For a
whitelist widening, the accuracy comparison above is the load-bearing evidence. Multi-node
deepep_v2 throughput for a different architecture is in #39080.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests. — the change is a one-entry gate widening; exercising it needs a deepep_v2 server on a blockwise-FP8 MoE checkpoint, which CI does not have.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed. — accuracy only, see above for why.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

I cannot add the run-ci label myself, so this needs someone with write access to start CI.

---

Independent of this change, but a deepep_v2 server needs it: #37211 — ep_scatter_from_psum
omits two positional kernel arguments, so any deepep_v2 prefill dies with
TypeError: dynamic_func() missing 2 required positional arguments before the effect of this
whitelist entry is observable. All numbers above were measured with that fix applied. Note
there are now three open PRs for the same one-line fix — #37211, #38518, #39399 — worth
deduplicating.

Touches the same tuple as #39080 ("MiMoV2ForCausalLM"); the two conflict textually and are
trivially rebasable in either order.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ❌ Run #34833429638
Latest PR Test (Extra): ❌ Run #34833429452
Latest PR Test (AMD ROCm 10): ❌ Run #34833429450