[BugFix] Support online dense model DP without overhead

[njhill]

Currently, there's unnecessary overhead when running non-MoE models in a data parallel configuration because the steps across the ranks are synchronized with redundant all-reduce ops and coordination is done to ensure "idle" ranks perform dummy forward passes.

This PR changes the parallel config at the worker level to be equivalent to DP=1 for non-MoE models, so each rank operates independently. When internal load-balancing is used, the DP coordinator still runs to propagate stats back from the engines for load balancing purposes, but the step/wave synchronization logic is disabled.

Fixes #24461.
Fixes #30655.

This is supported in the online / AsyncLLM case only.

The offline DP will now fail during startup for non-MoE models (it really makes no sense to use it in that configuration).

Benchmark on 4xH100:

vllm serve Qwen/Qwen3-8B --data-parallel-size 4 --uvicorn-log-level=error

vllm bench serve \
    --backend vllm \
    --model Qwen/Qwen3-8B \
    --dataset-name random \
    --random-input-len 128 \
    --random-output-len 512 \
    --ignore-eos \
    --port 8033 \
    --num-prompts 4000 \
    --max-concurrency 200 \
    --seed 42

Before

============ Serving Benchmark Result ============
Successful requests:                     4000      
Failed requests:                         0         
Maximum request concurrency:             200       
Benchmark duration (s):                  104.41    
Total input tokens:                      512000    
Total generated tokens:                  2048000   
Request throughput (req/s):              38.31     
Output token throughput (tok/s):         19615.26  
Peak output token throughput (tok/s):    21597.00  
Peak concurrent requests:                400.00    
Total token throughput (tok/s):          24519.08  
---------------Time to First Token----------------
Mean TTFT (ms):                          131.78    
Median TTFT (ms):                        124.31    
P99 TTFT (ms):                           404.36    
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          9.93      
Median TPOT (ms):                        9.95      
P99 TPOT (ms):                           10.08     
---------------Inter-token Latency----------------
Mean ITL (ms):                           9.93      
Median ITL (ms):                         9.79      
P99 ITL (ms):                            15.93     
==================================================

After

============ Serving Benchmark Result ============
Successful requests:                     4000      
Failed requests:                         0         
Maximum request concurrency:             200       
Benchmark duration (s):                  99.24     
Total input tokens:                      512000    
Total generated tokens:                  2048000   
Request throughput (req/s):              40.31     
Output token throughput (tok/s):         20636.52  
Peak output token throughput (tok/s):    22454.00  
Peak concurrent requests:                400.00    
Total token throughput (tok/s):          25795.66  
---------------Time to First Token----------------
Mean TTFT (ms):                          88.94     
Median TTFT (ms):                        74.67     
P99 TTFT (ms):                           379.48    
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          9.50      
Median TPOT (ms):                        9.50      
P99 TPOT (ms):                           9.66      
---------------Inter-token Latency----------------
Mean ITL (ms):                           9.50      
Median ITL (ms):                         9.38      
P99 ITL (ms):                            12.86     
==================================================

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

[gemini-code-assist]

Code Review

This pull request introduces a significant optimization for running dense (non-MoE) models in a data-parallel configuration by removing unnecessary synchronization overhead. The core idea is to treat each data-parallel rank as an independent worker for dense models, effectively setting their data-parallel size to 1 at the worker level. This avoids redundant all-reduce operations and complex wave synchronization, which are only necessary for MoE models. The DP coordinator's role is intelligently adapted: for dense models with internal load balancing, it continues to run for statistics propagation, but with wave coordination disabled. For external load balancing, it's disabled entirely for dense models. The changes are well-structured, with clear separation of concerns. The introduction of data_parallel_index to preserve the original rank is a clean solution. The related configurations and tests, especially the new test_needs_dp_coordination, are thorough and correctly validate the new logic. Overall, this is a solid improvement that should enhance performance for a common use case.

[mgoin]

@njhill do you think we should consider automatically setting api_server_count as we scale dp? For instance in your benchmarks you seemed to use 1/2 dp size