Repository navigation
Conversation
Replace hardcoded torch.cuda.* calls in the async output builder with the torch.accelerator.* / generic torch.Event / torch.Stream equivalents that the repo already mandates (pyproject banned-api; PRs vllm-project#3365, vllm-project#4496). This unblocks the AR model-runner warm-up path on XPU (and other non-CUDA accelerators) without relying on the XPU torch.cuda shim. Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
xuechendi
reviewed
Aug 19, 2026
| def __init__( | ||
| self, | ||
| payload: Any, | ||
| ready_event: torch.cuda.Event | None, |
Collaborator
There was a problem hiding this comment.
I think we have done monkey patch in vLLM side,
Please follow same method - https://github.com/vllm-project/vllm/blob/main/vllm/v1/worker/xpu_model_runner.py#L43
don't modify vllm_omni/worker/gpu_ar_model_runner.py
Collaborator
There was a problem hiding this comment.
Contributor
Author
There was a problem hiding this comment.
Reverted gpu_ar_model_runner.py and moved the fix into platforms/xpu/utils.py via torch_cuda_wrapper() (same pattern as vLLM xpu_model_runner.py). Added set_device and the missing stream/event/graph patches so the Omni async output path works on XPU without changing the shared GPU runner.
Align XPU cuda→xpu monkey patch with vLLM (streams, events, set_device, graph capture). Revert device-agnostic changes in gpu_ar_model_runner.py per platform-layer review feedback. Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com> Co-authored-by: Cursor <cursoragent@cursor.com>
xuechendi
approved these changes
Aug 20, 2026
fan2956
pushed a commit
to fan2956/vllm-omni
that referenced
this pull request
Aug 23, 2026
…oject#5569) Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com> Co-authored-by: Cursor <cursoragent@cursor.com>
fan2956
pushed a commit
to fan2956/vllm-omni
that referenced
this pull request
Aug 23, 2026
…oject#5569) Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com> Co-authored-by: Cursor <cursoragent@cursor.com>
AndyZhou952
pushed a commit
to AndyZhou952/vllm-omni
that referenced
this pull request
Aug 26, 2026
…oject#5569) Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com> Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
JoseCarlosGarcia95
pushed a commit
to valendra-tech/vllm-omni
that referenced
this pull request
Sep 5, 2026
…oject#5569) Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com> Co-authored-by: Cursor <cursoragent@cursor.com>
khairulkabir1661
pushed a commit
to khairulkabir1661/vllm-omni
that referenced
this pull request
Sep 25, 2026
…oject#5569) Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com> Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The AR async output builder in
gpu_ar_model_runner.pyuses hardcodedtorch.cuda.*(streams, events,set_device), which breaks the warm-up path on XPU and other non-CUDA accelerators.Fix
Replace them with the
torch.accelerator.*/ generictorch.Event/torch.StreamAPIs the repo already mandates :torch.cuda.current_stream()+with torch.cuda.stream(s)→torch.accelerator.current_stream()+torch.accelerator.set_stream()withtry/finallyrestore (same idiom asvllm_omni/diffusion/ipc.py)torch.cuda.Event/Stream→torch.Event/torch.Streamtorch.cuda.set_device()→torch.accelerator.set_device_index()No behavior change on CUDA; unblocks AR-stage models (e.g. Qwen3-TTS) on XPU.
Test
Qwen3-TTS-12Hz-1.7B runs to completion on Intel XPU with this change.