[Experimental][RFC] Eliminate redundant host copies from generated-video transport

[mglyn]

Summary

This draft adds an opt-in transport path that removes three redundant host-side copies after video generation:

Existing path                              Experimental path
-------------                              -----------------
GPU RGB tensor                             GPU RGB tensor
      | mandatory D2H                            | mandatory D2H
      v                                          | directly into registered SHM
pinned CPU buffer                                v
      | host memcpy [removed]              CUDA-registered POSIX SHM
      v                                          | borrowed view
POSIX SHM                                        v
      | host memcpy [removed]              NumPy / Torch view
      v                                          | borrowed RGB AVFrame
private CPU buffer                               v
      | RGB plane copy [removed]           PyAV encoder -> MP4
      v
PyAV encoder -> MP4

The implementation writes CUDA output directly into registered POSIX shared memory, preserves the mapping for the lifetime of the receiver's borrowed view, and uses VideoFrame.from_numpy_buffer for contiguous RGB uint8 frames. The mandatory device-to-host transfer remains; "zero copy" refers to the eliminated host-side copies around IPC and encoding.

The existing path remains the default. Enable the experiment with:

VLLM_OMNI_ZERO_COPY_SHM=1
VLLM_OMNI_VIDEO_BORROWED_FRAMES=1

The direct GPU-to-SHM direction was proposed earlier in upstream PR #2197, which removed the NumPy staging copy. This draft extends that early work by registering POSIX SHM for D2H, borrowing the receiver mapping, and borrowing RGB buffers in PyAV. It remains a small copy-elision baseline that can compose with the chunkwise producer/consumer design in RFC #6872.

Evaluation

Both experiments used separately started server blocks in A1 -> B1 -> B2 -> A2 order. Prompt, seed, model configuration, and commit were fixed; only the two environment flags differed.

Workload	Configuration	Samples per path
MiniMax-H3 T2VA	8x NVIDIA H200, USP8, text-encoder TP8, VAE tile PP8; 1344x768, 362 frames, 24 FPS, 2 denoising steps	10
LTX-2.5 T2VA	4x NVIDIA H200, USP4, VAE tile PP4; 1920x1088, 361 frames, 24 FPS	4

MiniMax-H3 end-to-end breakdown

Stage-0 includes model generation, Video-VAE decode, and output transport. Output-ready wait and the derived transport boundary are two views of that transport tail; they are not additional sequential stages.

Scope	Metric	Existing	Experimental	Change
Request	End-to-end latency, mean	10.714 s	10.262 s	-4.22%
Stage-0 total	Generation and transport, mean	9.634 s	9.150 s	-5.02%
↳ Model subphase	Video-VAE decode, median	1.968 s	1.968 s	-0.02%
↳ Transport view	Output-ready wait, mean	1.537 s	0.756 s	-50.83%
↳ Transport view	Derived transport boundary, mean	1.548 s	0.994 s	-35.77%
Finalization	MP4 encoding, mean	1.070 s	1.101 s	+2.91%

The request-level bootstrap 95% confidence interval for the mean end-to-end difference was [-0.620, -0.261] s. All 20 measured MP4 files were byte-identical. GPU memory was unchanged: 94,272 MiB profiler peak; 99,865 MiB existing versus 99,863 MiB experimental maximum sampled usage.

LTX-2.5 generation breakdown

The LTX run used native pipeline-profiler aggregation. Its Stage-0 subphases below are not exhaustive, but they show that the observed change is outside denoise and VAE decode.

Scope	Metric	Existing	Experimental	Change
Stage-0 total	Generation and transport, mean	28.653 s	25.797 s	-9.97%
↳ Model subphase	Low-resolution denoise, mean	4.329 s	4.331 s	+0.05%
↳ Model subphase	Refinement denoise, mean	11.688 s	11.719 s	+0.26%
↳ Model subphase	Video-VAE decode, mean	8.060 s	8.053 s	-0.08%

Client E2E, output-ready wait, derived transport, and MP4 encoding were not aggregated with the same request-level methodology as the MiniMax-H3 run, so they are intentionally not compared here. All eight measured requests succeeded. The two existing-path Stage-0 block means were 30.270 s and 27.037 s, making the 9.97% result directional rather than a stable point estimate.

Validation

python -m pytest -q tests/diffusion/test_diffusion_ipc.py tests/diffusion/test_ipc_async.py
28 passed

VLLM_OMNI_ZERO_COPY_SHM=1 python -m pytest -q tests/diffusion/test_diffusion_ipc.py tests/diffusion/test_ipc_async.py
28 passed