[Core] Avoid unnecessary coordination for non-MoE data parallel#24828

ZJY0516 wants to merge 4 commits intovllm-project:mainvllm-project/vllm:mainfrom

ZJY0516:moe-dpZJY0516/vllm:moe-dpCopy head branch name to clipboard