perf: rotate Model Runner V2 output copy streams

vllm_musa/patches/series/0172-MUSA-rotate-V2-runner-output-copy-streams.patch

@@ -0,0 +1,48 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: musa <musa@local>
+Date: Mon, 28 Sep 2026 12:00:00 +0800
+Subject: [PATCH] MUSA: rotate V2 runner output copy streams
+
+On MUSA, once step N+1's copy-stream wait is queued, step N's output copy
+event completes only after step N+1 finishes, so each sampled token is
+published one step late and the host launches every other step after the
+GPU has drained. Give each in-flight step its own output copy stream.
+
+diff --git a/vllm/v1/worker/gpu/model_runner.py b/vllm/v1/worker/gpu/model_runner.py
+--- a/vllm/v1/worker/gpu/model_runner.py
++++ b/vllm/v1/worker/gpu/model_runner.py
+@@ -192,6 +192,13 @@
+         self.is_encoder_decoder = self.model_config.is_encoder_decoder
+
+         self.output_copy_stream = torch.cuda.Stream(self.device)
++        # MUSA: a copy-stream wait queued for step N+1 delays step N's copy event
++        # until step N+1 finishes, so each in-flight step gets its own stream.
++        self._output_copy_streams = [
++            self.output_copy_stream,
++            torch.cuda.Stream(self.device),
++        ]
++        self._output_copy_stream_idx = 0
+
+         # Pipeline parallelism.
+         self.use_pp = self.parallel_config.pipeline_parallel_size > 1
+@@ -1818,7 +1825,7 @@
+             sampler_output=sampler_output,
+             num_sampled_tokens=num_sampled,
+             main_stream=self.main_stream,
+-            copy_stream=self.output_copy_stream,
++            copy_stream=self._next_output_copy_stream(),
+             check_ep_fault=self.check_ep_fault,
+             routed_experts=routed_experts,
+         )
+@@ -1894,6 +1901,11 @@
+
+         return async_output
+
++    def _next_output_copy_stream(self) -> torch.cuda.Stream:
++        streams = self._output_copy_streams
++        self._output_copy_stream_idx = (self._output_copy_stream_idx + 1) % len(streams)
++        return streams[self._output_copy_stream_idx]
++
+     def take_draft_token_ids(self) -> DraftTokenIds | None:
+         return self.draft_tokens_handler.get_draft_tokens()
+

vllm_musa/patches/series/README.md

@@ -17,7 +17,7 @@ is pre-patched.
   Author headers are normalized to the synthetic
   `musa <musa@local>` identity.
 
-Currently **171 patches**. This branch includes the Qwen3.6 patches for common
+Currently **172 patches**. This branch includes the Qwen3.6 patches for common
 GDN decode metadata reuse, uniform-decode SSM slot-mapping removal, and the
 BF16 W1 tile specialization, plus the contract-bound DeepSeek-V4 MTP
 sparse-prefill headroom and mixed-prefill queue-fence patches. It additionally
@@ -46,7 +46,9 @@ shape fallback. On that same native path the compressor also writes
 packed kv/score+ape into the state cache so Triton `save_partial_states`
 is skipped. Interleaved MRoPE rebuilds the T/H/W frequency layout with a
 strided copy rather than per-channel index arithmetic, which keeps the
-per-layer launch count off the eager prefill path. The fused TileLang `hc_head` is
+per-layer launch count off the eager prefill path. Model Runner V2 rotates two
+output copy streams so a step's sampled tokens are not held back by the next
+step's copy-stream wait. The fused TileLang `hc_head` is
 enabled on MUSA by importing TileLang before the eager JIT decorators capture
 their module globals. DeepEP shutdown now drops cached handles before native
 teardown and supports both explicit `destroy()` and legacy destructor-only