Skip to content

vllm v0.26.0 compat - #97

Merged
yeahdongcn merged 3 commits into
MooreThreads:mainfrom
Joey-gvwal:v0.26.0-compat
Jul 30, 2026
Merged

yeahdongcn merged 3 commits into
MooreThreads:mainfrom
Joey-gvwal:v0.26.0-compat

Conversation

@Joey-gvwal

Copy link
Copy Markdown
Contributor

Summary

vLLM v0.26.0 (via vllm-musa) touches two torch surfaces that torch_musa 2.9
does not provide as-is:

  1. torch.accelerator.get_memory_info() — a torch 2.10+ API, absent on
    torch_musa 2.9 (engine-core startup hits AttributeError).
  2. The kernels vLLM relocated into csrc/libtorch_stable/ use upstream CUDA
    spellings (cudaDeviceProp, cudaStream_t, cublasHandle_t,
    aoti_torch_get_current_cuda_stream, …), but torch_musa's stable ABI only
    exposes the musa* names.

These shims let both build/run unchanged on MUSA, keeping the CUDA→MUSA mapping
in torchada instead of downstream forks.

Changes

  • _patch.py: map torch.accelerator.get_memory_info →
    torch.musa.mem_get_info (same (free, total) contract).
  • csrc/stable_compat/: add #ifndef-guarded CUDA→MUSA aliases so
    libtorch-stable kernels compile against upstream names —
    • cuda_runtime.h: cudaDeviceProp, cudaError_t, cudaSuccess,
      cudaStream_t, cudaGet{DeviceCount,DeviceProperties,Device,ErrorString}
    • cublas_v2.h: cublasHandle_t → mublasHandle_t
    • torchada_stable_box.h: forward aoti_torch_get_current_cuda_stream →
      aoti_torch_get_current_musa_stream; define TORCH_UTILS_CHECK
      Guards make each alias a no-op where an mcc porting layer already provides
      the name.

Testing

Rebuilt vllm-musa's stable extension (_C_stable_libtorch) against this
torchada on MTT S5000 (torch_musa 2.9.1). All 4 extensions import; smokes pass:
FP8 quant (Qwen3-8B/32B-FP8), MLA (DeepSeek-V2-Lite-FP8, FLASHMLA), and
multi-GPU stable custom_all_reduce (Qwen3-32B-FP8, tp2).

@yeahdongcn yeahdongcn changed the title V0.26.0 compat vllm v0.26.0 compat Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants