[fzyzcjy]

Motivation

For example, suppose we want to run DeepSeek-V3 on 8xH200. Before this change, it takes ~120s to warmup all 8 GPUs, because each GPU warmup are done serially, which is converted into parallel operations.

Modifications

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[zhyncs]

it takes ~120s to warmup all 8 GPUs

How long will it take after this change?

[fzyzcjy]

@zhyncs Briefly checked the logs, looks like 138s -> 54s. In addition, I am wondering where larger-scale deployment, say 2x8xH100, will be affected by this more, because the original code will need to warmup 16 cards one by one (not done any experiments though).

Before

[2025-03-23 08:39:08] INFO:     Started server process [874108]
[2025-03-23 08:41:26] The server is fired up and ready to roll!

After

[2025-03-23 09:08:22] INFO:     Started server process [893444]
[2025-03-23 09:09:16] The server is fired up and ready to roll!