[Bugfix] Support MTP speculative decoding with pipeline parallelism on the PD producer side

[gjc0824]

Purpose

Enable MTP (Multi-Token Prediction) and Eagle3 speculative decoding to run when the target model is sharded with Pipeline Parallelism (PP > 1) in PD-disaggregated (prefill/decode split) deployments. Previously this combination failed because several code paths treated the drafter as if it were partitioned across pipeline stages like the target model, when in fact local drafters (Eagle, MTP) are loaded only on the last PP rank with an effective PP size of 1.

Root cause & fixes:

• Config rejected valid draft models (vllm/config/model.py): ModelConfig.verify_with_parallel_config raised NotImplementedError("Pipeline parallelism is not supported...") for Eagle/MTP drafters, since it checked the target-model PP registry for any model under PP > 1. Drafters run locally on the last rank, so this check is now skipped when runner_type == "draft".

• Drafter layer count was wrong under PP (deepseek_eagle3.py, llama_eagle3.py): Eagle3 drafters computed target_layer_num via get_num_layers(parallel_config), which returns the per-rank shard size. Under PP the drafter (on the last rank) only saw a fraction of the layers, corrupting layer_types indexing. Switched to get_total_num_hidden_layers().

• Qwen3.5 MTP forward assumed PP intermediates (qwen3_5_mtp.py): forward branched on is_first_rank and read IntermediateTensors on non-first ranks. Because the MTP drafter is local to the last rank, it now always combines token embeddings with the target hidden states and never consumes PP intermediates from previous stages.

• Drafter initialized on every PP rank (vllm/v1/worker/gpu_model_runner.py): drafter attention-backend and cudagraph-dispatcher init ran on all ranks even though only the last rank loads the drafter. Both are now guarded by get_pp_group().is_last_rank.

• Unsafe on the decode side (vllm/config/vllm.py): added _validate_spec_decode_pp_config() — MTP + PP > 1 is permitted only on the prefill (producer) side (kv_role='kv_producer') of a PD deployment, since the decode-side verification loop cannot coordinate draft-token propagation across pipeline stages. A clear ValueError is raised otherwise, guiding users toward data parallelism + MTP on the decode side.

Non-invasive: PP = 1 and non-speculative-decode code paths are completely unaffected.

Test Plan

Behavior is verified end-to-end (no repo unit tests added). Validate with a PD-disaggregated serving setup:

Producer (prefill) — PP = 2 + MTP:

vllm serve <TARGET_MODEL> \
  --pipeline-parallel-size 2 --tensor-parallel-size 1 \
  --speculative-config '{"method":"mtp","model":{"type":"qwen3_5_mtp","model":"<DRAFTER_MODEL>"}}' \
  --kv-transfer-config '{"role":"producer","kv_role":"kv_producer","kv_connector":"<CONNECTOR>","engine_id":0}'

Consumer (decode) — PP = 1:

vllm serve <TARGET_MODEL> \
  --kv-transfer-config '{"role":"consumer","kv_role":"kv_consumer","kv_connector":"<CONNECTOR>","engine_id":1}'

Negative config checks:

• MTP + PP = 2 without kv_producer → expect the new ValueError.
• Eagle3 drafter model under PP = 2 → no longer hits the PP-support NotImplementedError.

Test Result

Functional verification (deterministic from the code paths):

• Config: MTP + PP > 1 with kv_producer → accepted; MTP + PP > 1 without a producer role → raises the documented ValueError.
• Eagle3 drafter under PP = 2 → passes model-config validation (previously raised NotImplementedError).
• Drafter attention-backend / cudagraph resources are initialized only on the last PP rank; non-last ranks no longer error during init.
• target_layer_num reflects the total hidden-layer count (not the per-rank shard), giving correct layer_types indexing in the Eagle3 drafter.

End-to-end PD serving (PP = 2 + MTP on the producer) produced correct outputs:

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @gjc0824.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork