[wangyicong52]

Overview

Follow-up to #59857, enabling text-only DeepSeek-V4.1-Flash PCP Prefill with NIXL P/D. Stacked on #59325 for replicated-PCP producer completion.

Claims

• Allow GPU NixlConnector pull producers to use PCP, with PCP=1 on Decode.
• Allow fixed-length DSpark and optional APC with SWA bounded replay.
• Retain PP=DP=DCP=1 and synchronous scheduling. Consumer PCP, push, bidirectional transfer and custom connector paths remain unsupported.

Validation

Image

vllm/vllm-openai:nightly-edad8b03c5285a1c12e7e4f83c1af709b319a01b

Official linux/amd64 digest: sha256:d31ecffc8a9613b1f7c6c849a45e01bb9f0ac86a177f7dd802f29b4e51390198.

Software

• Overlaid this PR at commit 20a882654abbb8aee6ebef1ba110a76314d8a164 (vllm/models/deepseek_v41/nvidia/model.py).
• Overlaid the pull-mode completion fix from [Bugfix][NIXL] Stop clearing finished_sending for replicated-PCP producer ranks #59325 at commit 52c8468f7c27a73047460376b5f90fcb13af8695 (vllm/distributed/kv_transfer/kv_connector/v1/nixl/connector.py).

vLLM startup

Relevant environment:

export VLLM_USE_V2_MODEL_RUNNER=1
export VLLM_NIXL_SIDE_CHANNEL_HOST=<LOCAL_ROLE_HOST>
export VLLM_NIXL_SIDE_CHANNEL_PORT=5600
export UCX_TLS=all
export UCX_NET_DEVICES=<RDMA_DEVICES>
export NCCL_IB_DISABLE=1
export NCCL_SOCKET_IFNAME=<SOCKET_IFACE>
export GLOO_SOCKET_IFNAME=<SOCKET_IFACE>

Prefill (TP2/PCP4):

vllm serve <MODEL_PATH> \
  --host 0.0.0.0 \
  --port 18001 \
  --served-model-name deepseek-v4.1-flash \
  --trust-remote-code \
  --language-model-only \
  --tokenizer-mode deepseek_v41 \
  --reasoning-parser deepseek_v41 \
  --distributed-executor-backend mp \
  --attention-backend FLASHMLA_SPARSE_DSV41 \
  --kv-cache-dtype fp8_ds_mla \
  --block-size 64 \
  --tensor-parallel-size 2 \
  --pipeline-parallel-size 1 \
  --data-parallel-size 1 \
  --decode-context-parallel-size 1 \
  --prefill-context-parallel-size 4 \
  --enable-expert-parallel \
  --all2all-backend allgather_reducescatter \
  --engram-config '{"cpu_offload":true}' \
  --moe-backend humming \
  --max-model-len 131072 \
  --max-num-batched-tokens 4096 \
  --max-num-seqs 64 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --swa-bounded-replay \
  --speculative-config '{"method":"dspark","num_speculative_tokens":5}' \
  --enforce-eager \
  --no-async-scheduling \
  --disable-uvicorn-access-log \
  --kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_producer","kv_buffer_device":"cuda","kv_load_failure_policy":"fail"}'

Decode (TP8/PCP1):

vllm serve <MODEL_PATH> \
  --host 0.0.0.0 \
  --port 18002 \
  --served-model-name deepseek-v4.1-flash \
  --trust-remote-code \
  --language-model-only \
  --tokenizer-mode deepseek_v41 \
  --reasoning-parser deepseek_v41 \
  --distributed-executor-backend mp \
  --attention-backend FLASHMLA_SPARSE_DSV41 \
  --kv-cache-dtype fp8_ds_mla \
  --block-size 64 \
  --tensor-parallel-size 8 \
  --pipeline-parallel-size 1 \
  --data-parallel-size 1 \
  --decode-context-parallel-size 1 \
  --prefill-context-parallel-size 1 \
  --enable-expert-parallel \
  --all2all-backend allgather_reducescatter \
  --engram-config '{"cpu_offload":true}' \
  --moe-backend humming \
  --max-model-len 131072 \
  --max-num-batched-tokens 2048 \
  --max-num-seqs 64 \
  --gpu-memory-utilization 0.90 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --swa-bounded-replay \
  --speculative-config '{"method":"dspark","num_speculative_tokens":5}' \
  --enforce-eager \
  --no-async-scheduling \
  --disable-uvicorn-access-log \
  --kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_consumer","kv_buffer_device":"cuda","kv_load_failure_policy":"fail"}'

GSM8K

EvalScope version: 1.11.1.

evalscope eval \
  --eval-type openai_api \
  --api-url http://<ROUTER_HOST>:30000/v1 \
  --api-key EMPTY \
  --model deepseek-v4.1-flash \
  --model-id deepseek-v4.1-flash \
  --datasets gsm8k \
  --dataset-hub modelscope \
  --dataset-args '{"gsm8k":{"few_shot_num":4}}' \
  --seed 42 \
  --eval-batch-size 64 \
  --generation-config '{"temperature":0.0,"top_p":1.0,"max_tokens":8192,"reasoning_effort":"high","seed":42,"timeout":10000}' \
  --timeout 10000 \
  --work-dir <WORK_DIR> \
  --no-timestamp

• Result: 1283 / 1319 correct (97.2707%), within the local historical control range of 1279-1286 / 1319.
• Execution: 1319 / 1319 requests succeeded with 0 API errors; 3 responses reached the 8192-token limit.
• Truncation diagnostic: a separate 65536-token replay recovered correct final answers for 2 of the 3 truncated samples; the primary score was not replaced.
• NIXL: 10,552 successful rank transfers and 53,282,008,576 transferred bytes, with 0 failed transfers, notification failures, or KV expirations.
• Health: 0 preemptions and 0 service restarts.
• Cache-state note: prefix-cache reset endpoints were unavailable because the servers were not launched in dev mode; both roles were idle with zero active KV usage before the run.

Details

Keep the capability checks in the NVIDIA model and reuse the existing cache-transfer path. Add one focused PCP+NIXL configuration test. The completion fix is inherited from #59325.

AI assistance was used for implementation and tests.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[mergify]

Documentation preview: https://vllm--61746.org.readthedocs.build/en/61746/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wangyicong52.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wangyicong52.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wangyicong52.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wangyicong52.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork