[bugfix][async scheduling] fix extra cuda context in device 0 with EP/DP

[youkaichao]

Purpose

See https://forums.developer.nvidia.com/t/when-a-thread-has-a-primary-cuda-context-does-the-child-thread-it-creates-automatically-inherit-the-cuda-context/362810 , a new thread does not have any cuda context, and later cuda runtime call might create a context in device 0.

Test Plan

Run vLLM serve with EP/DP:

vllm serve Qwen/Qwen3-30B-A3B-Instruct-2507 -dp 2 -ep --port 8899

test with multiple requests:

vllm bench serve \
  --backend openai \
  --base-url http://127.0.0.1:8899 \
  --model Qwen/Qwen3-30B-A3B-Instruct-2507 \
  --dataset-name random \
  --num-prompts 100 \
  --input-len 1024 \
  --output-len 256

Test Result

Before the fix, after the benchmark script, worker 1 will take around 800 MiB memory on device 0.

After the fix, each worker only resides on one GPU.

[youkaichao]

move this part of code here, to be after self.worker.init_device(), so that self.worker.device is initialized properly.

[gemini-code-assist]

Code Review

This pull request addresses a critical issue where asynchronous scheduling threads could implicitly create an extra CUDA context on device 0, leading to unnecessary memory consumption, especially in Expert Parallel (EP) and Data Parallel (DP) setups. The changes correctly relocate the asynchronous output copy thread initialization to ensure the worker is fully loaded before the thread starts. Additionally, the async_output_busy_loop method now explicitly sets the CUDA device for the thread to match the worker's assigned device, preventing the creation of unintended contexts. This is a well-targeted fix that directly resolves the described bug and improves resource management.

[njhill]

Nice find, thanks @youkaichao!