[BugFix][DP] Send DP wave completion only from dp_rank==0

We were sending this notification back to the front-end for local_dp_rank==0 rather than dp_rank==0 which could result in redundant messages.

Since the wave count is included in the message explicitly, this doesn't actually cause any functional issues.

Co-authored-by: kourosh hakhamaneshi <[email protected]>
Signed-off-by: Nick Hill <[email protected]>

Author: njhill

vllm/v1/engine/core.py

@@ -701,7 +701,7 @@ def _init_data_parallel(self, vllm_config: VllmConfig):
             for i in range(local_dp_rank * world_size, (local_dp_rank + 1) *
                            world_size))
 
-        self.local_dp_rank = local_dp_rank
+        self.dp_rank = dp_rank
         self.dp_group = vllm_config.parallel_config.stateless_init_dp_group()
         self.current_wave = 0
 
@@ -774,7 +774,7 @@ def run_busy_loop(self):
                 local_unfinished_reqs)
 
             if not self.engines_running:
-                if self.local_dp_rank == 0:
+                if self.dp_rank == 0:
                     # Notify client that we are pausing the loop.
                     logger.debug("Wave %d finished, pausing engine loop.",
                                  self.current_wave)