[CI Bug] Fully solve accuracy issue for DSv3.2 + MTP + Sequence Parallel

[yewentao256]

Purpose

A through fix for #47902

Test

vllm serve deepseek-ai/DeepSeek-V3.2 --trust-remote-code --kernel-config.enable_flashinfer_autotune=False --enable-expert-parallel --tensor-parallel-size 8 --tokenizer-mode deepseek_v32 --tool-call-parser deepseek_v32 --enable-auto-tool-choice --reasoning-parser deepseek_v3 --port 9256 --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

lm_eval --model local-completions --model_args "base_url=http://127.0.0.1:9256/v1/completions,model=$MODEL,num_concurrent=1024" --tasks gsm8k

Now (with SP enabled)

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.9606|±  |0.0054|
|     |       |strict-match    |     5|exact_match|↑  |0.9598|±  |0.0054|

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[yewentao256]

CC @mgoin

[drakosha]

AI assisted:

Independent confirmation from another model family, plus verification of this fix: the same root cause also produces a hard crash (not just an accuracy drop) — a device-side assert that kills all workers on the first generation request.

On GLM-5.2 (GlmMoeDsaForCausalLM, same DeepSeekMTP draft path) with --enable-expert-parallel + TP>1 + MTP, whenever use_sequence_parallel_moe is active the MTP block returns reduce-scattered rows (ceil(T_pad / tp)), nothing re-gathers them, and the proposer then indexes with global token positions:

/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:111: operator(): Assertion `-sizes[i] <= index && index < sizes[i] && "index out of bounds"` failed.

at vllm/v1/spec_decode/llm_base_proposer.py:605 (sample_hidden_states = last_hidden_states[token_indices_to_sample]). On current main this affects DP>1 + EP + TP>1 + MTP deployments (the #47902 gate hides it at DP=1); we hit it at DP=1 via an image whose parallel.py predates #47902. Same signature was independently reported by a tester on #46514 (TP8 + EP + MTP k=4, GLM-5.2-AWQ, no DCP).

Verified this PR's deepseek_mtp.py change on 4×H200 NVL, GLM-5.2-NVFP4, TP4 + DCP4 + EP + fp8_ds_mla + MTP k=1, with SP-MoE active (DP gate removed, matching this PR's parallel.py): before — deterministic crash on the first request, the draft forward returns 5 rows for a 19-token prefill (instrumented); after — full 19 rows, no assert, coherent output across prefill/decode/verify steps.