Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ The plugin leverages the following key components:

| vLLM Version | PyTorch Version | Engine | Status |
|--------------|-----------------|---------|--------------|
| v0.24.0 | 2.9.x | V1 only | ✅ Supported |
| v0.24.0 | 2.11.x | V1 only | ✅ Supported |

> **Note**: This plugin uses vLLM's V1 engine architecture (the V0 engine is not supported). Within the V1 engine, vLLM v0.24.0 auto-selects its **Model Runner V2** for certain architectures (e.g. Qwen3, DeepSeek-V2, Llama) and the V1 model runner for others; both are supported on MUSA. Set `VLLM_USE_V2_MODEL_RUNNER=1` or `0` to force one.

Expand Down Expand Up @@ -79,7 +79,7 @@ install below.

Most of the MUSA wheels are not published on public PyPI, so installing
`requirements/musa.txt` with pip's default index fails with
`No matching distribution found for torch==2.9.1.post1+musa5.2.0s5000`.
`No matching distribution found for torch==2.11.0.post1+musa5.2.0`.

The MUSA wheels must be **resolved** with the Moore Threads index as the sole
`--index-url`, which is why the install below starts with a pass that installs
Expand Down
4 changes: 2 additions & 2 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@

| vLLM 版本 | PyTorch 版本 | 引擎 | 状态 |
|-----------|--------------|---------|--------------|
| v0.24.0 | 2.9.x | 仅 V1 | ✅ 已支持 |
| v0.24.0 | 2.11.x | 仅 V1 | ✅ 已支持 |

> **注意**:本插件使用 vLLM 的 V1 引擎架构(不支持 V0 引擎)。在 V1 引擎内部,vLLM v0.24.0 会为部分架构(如 Qwen3、DeepSeek-V2、Llama)自动选用 **Model Runner V2**,其余架构使用 V1 model runner;两者在 MUSA 上均受支持。设置 `VLLM_USE_V2_MODEL_RUNNER=1` 或 `0` 可强制指定其一。

Expand Down Expand Up @@ -76,7 +76,7 @@ MUSA SDK 的主机上,请使用下面的源码安装。

大部分 MUSA wheel 并未发布到公共 PyPI,因此用 pip 默认索引安装
`requirements/musa.txt` 会失败并报
`No matching distribution found for torch==2.9.1.post1+musa5.2.0s5000`。
`No matching distribution found for torch==2.11.0.post1+musa5.2.0`。

MUSA wheel 必须在摩尔线程索引作为**唯一** `--index-url` 的前提下解析,因此下面的
安装步骤以“只装 MUSA wheel”的一步开始。
Expand Down
16 changes: 8 additions & 8 deletions docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ bash docker/build_image.sh
With the defaults this produces:

```
vllm-musa:ubuntu22.04_py3.10_musa_runtime_5.2_pytorch_release_2.9.1.post1_musa5.2.0s5000
vllm-musa:ubuntu22.04_py3.10_musa_runtime_5.2_pytorch_release_2.11.0.post1_musa5.2.0
```

The image accepts the model and engine arguments directly:
Expand Down Expand Up @@ -180,7 +180,7 @@ Two build-time details make this work on such a host:
```bash
docker run --rm <MUSA GPU flags> \
--entrypoint python \
vllm-musa:ubuntu22.04_py3.10_musa_runtime_5.2_pytorch_release_2.9.1.post1_musa5.2.0s5000 \
vllm-musa:ubuntu22.04_py3.10_musa_runtime_5.2_pytorch_release_2.11.0.post1_musa5.2.0 \
-c "import torch, torch_musa; print('musa available:', torch.musa.is_available())"
```

Expand Down Expand Up @@ -218,12 +218,12 @@ for the validated runtime flags.
9. **vllm-openai** — the default serving target, with `vllm serve` as its
entrypoint.

The Triton `3.2.0` pin is intentional for the torch `2.9.1` MUSA stack. Torch
Inductor reads `KernelMetadata.cluster_dims`; the MUSA backend in Triton `3.6.0`
does not expose that field, while the `3.2.0` `mtgpu` backend does. The explicit
vLLM runtime dependency pass also restores the `fastapi[standard]` extras and
`pycountry` that are skipped when the vendored requirements are installed with
`--no-deps` to protect the MUSA torch pins.
The Triton `3.2.0` pin is intentional for the MUSA 5.2 stack and is maintained
independently of the PyTorch release. Validate image imports, Inductor, and
compiled serving before changing it. The explicit vLLM runtime dependency pass
also restores the `fastapi[standard]` extras and `pycountry` that are skipped
when the vendored requirements are installed with `--no-deps` to protect the
MUSA torch pins.

For the reasoning behind the pip-index split and the runtime shims, see the
comments in `docker/musa.Dockerfile`.
29 changes: 21 additions & 8 deletions docker/musa.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ RUN apt-get update && \
python -m pip install --upgrade pip && \
rm -rf /var/lib/apt/lists/*

# The torch 2.9.x MUSA wheel links MKL with .so.2 sonames. Ubuntu 22.04 apt
# The MUSA PyTorch wheel links MKL with .so.2 sonames. Ubuntu 22.04 apt
# ships the same logical MKL components without that suffix.
RUN ln -sf /usr/lib/x86_64-linux-gnu/libmkl_intel_lp64.so \
/usr/lib/x86_64-linux-gnu/libmkl_intel_lp64.so.2 && \
Expand Down Expand Up @@ -302,20 +302,31 @@ RUN printf '%s\n' \
'' \
'def requirement_prefix(dist_name):' \
' pattern = re.compile(rf"^{re.escape(dist_name)}==(.+)$")' \
' for line in Path("requirements/musa_private.txt").read_text().splitlines():' \
' match = pattern.match(line.strip())' \
' if match:' \
' return match.group(1).split("*", 1)[0]' \
' raise RuntimeError(f"missing {dist_name} pin in requirements/musa_private.txt")' \
' requirement_files = (' \
' "requirements/musa_private.txt",' \
' "requirements/common.txt",' \
' )' \
' for requirement_file in requirement_files:' \
' for line in Path(requirement_file).read_text().splitlines():' \
' match = pattern.match(line.strip())' \
' if match:' \
' return match.group(1).split("*", 1)[0]' \
' raise RuntimeError(f"missing {dist_name} pin in {requirement_files}")' \
'' \
'exact_version_dists = frozenset({"torchada", "torch", "torch_musa", "torchvision", "torchaudio", "deep_ep"})' \
'' \
'expected = (' \
' ("torchada", "torchada", requirement_prefix("torchada")),' \
' ("numpy", "numpy", "1.26."),' \
' ("torch", "torch", requirement_prefix("torch")),' \
' ("torch_musa", "torch_musa", requirement_prefix("torch_musa")),' \
' ("torchvision", "torchvision", requirement_prefix("torchvision")),' \
' ("torchaudio", "torchaudio", requirement_prefix("torchaudio")),' \
' ("mate", "mate", ""),' \
' ("flash_attn_3", "flash_attn_3", ""),' \
' ("flash_mla", "flash_mla", ""),' \
' ("deep-gemm", "deep_gemm", ""),' \
' ("deep_ep", "deep_ep", requirement_prefix("deep_ep")),' \
' ("tilelang_musa", "tilelang", ""),' \
' ("triton", "triton", requirement_prefix("triton")),' \
' ("uvloop", "uvloop", ""),' \
Expand All @@ -328,11 +339,13 @@ RUN printf '%s\n' \
'for dist_name, module_name, prefix in expected:' \
' importlib.import_module(module_name)' \
' installed = version(dist_name)' \
' if prefix and not installed.startswith(prefix):' \
' if dist_name in exact_version_dists and installed != prefix:' \
' raise RuntimeError(f"{dist_name} expected exactly {prefix}, got {installed}")' \
' if dist_name not in exact_version_dists and prefix and not installed.startswith(prefix):' \
' raise RuntimeError(f"{dist_name} expected {prefix}, got {installed}")' \
' print(f"PASS import {module_name} version={installed}")' \
'' \
'for module_name in ("torchada", "vllm", "vllm_musa"):' \
'for module_name in ("vllm", "vllm_musa"):' \
' module = importlib.import_module(module_name)' \
' print("PASS import %s version=%s" % (module_name, getattr(module, "__version__", "unknown")))' \
> /tmp/vllm_musa_import_check.py && \
Expand Down
5 changes: 3 additions & 2 deletions requirements/common.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Runtime Python dependencies that are portable across MUSA images.
# 0.1.76 ports nested torch/cuda.h includes to torch/musa.h during builds.
torchada>=0.1.76
# 0.1.79 contains the torch 2.11-compatible streams path used by compiled
# vLLM-MUSA.
torchada==0.1.79
mthreads-ml-py>=2.4.1
numpy==1.26
openai>=2.24.0
Expand Down
14 changes: 7 additions & 7 deletions requirements/musa_private.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,19 @@
# projects on public PyPI, so they must resolve from the Moore Threads index
# only. Versions below track the MUSA release_5.2.0 Python Wheel guide.

torch==2.9.1.post1+musa5.2.0
torch_musa==2.9.1.post1+musa5.2.0
torchvision==0.24.1.post1+musa5.2.0
torchaudio==2.9.1+musa5.2.0
torch==2.11.0.post1+musa5.2.0
torch_musa==2.11.0.post1+musa5.2.0
torchvision==0.26.0.post1+musa5.2.0
torchaudio==2.11.0+musa5.2.0
mate==0.2.4
flash_attn_3==0.2.4
flash_mla==0.2.4
deep-gemm==0.2.4
sageattention==0.2.4
deep_ep==1.1.0+musa5.2.0torch2.9.1.post1s5000
deep_ep==1.1.0+musa5.2.0torch2.11.0.post1
tilelang_musa==0.1.8+musa.3
# torch 2.9.1 Inductor reads KernelMetadata.cluster_dims. The MUSA backend in
# Triton 3.6.0 does not expose it; the 3.2.0 mtgpu backend does.
# The MUSA 5.2.0 Triton backend remains pinned independently of the PyTorch
# release; verify this pin in the 2.11 image before changing it.
triton==3.2.0
apache-tvm-ffi==0.1.9.post3+musa.1
torch-c-dlpack-ext==0.1.5
1 change: 1 addition & 0 deletions requirements/test.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
# Test runner included in the Docker image.
pytest
pytest-asyncio
33 changes: 33 additions & 0 deletions tests/test_build_ccache.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,41 @@
import subprocess
from pathlib import Path

import pytest

from build_utils.ccache import configure_compiler_cache

_CONFIGURE_ENV_KEYS = (
"PATH",
"CXX",
"PYTORCH_MCC",
"CCACHE_DIR",
"CCACHE_BASEDIR",
"CCACHE_COMPILERCHECK",
"CCACHE_NOHASHDIR",
"CCACHE_SLOPPINESS",
"CCACHE_MAXSIZE",
"VLLM_MUSA_REAL_CCACHE",
"VLLM_MUSA_REAL_MCC",
"VLLM_MUSA_CCACHE_MUSA_COMPILER",
"VLLM_MUSA_CCACHE_SOURCE_DIR",
)


@pytest.fixture(autouse=True)
def _restore_compiler_cache_environment():
before = {
name: os.environ[name]
for name in _CONFIGURE_ENV_KEYS
if name in os.environ
}
yield
for name in _CONFIGURE_ENV_KEYS:
if name in before:
os.environ[name] = before[name]
else:
os.environ.pop(name, None)


def _write_executable(path: Path, content: str) -> None:
path.write_text(content, encoding="utf-8")
Expand Down
3 changes: 3 additions & 0 deletions tests/test_deepseek_v4_swiglu_fp8_fusion_source.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,14 @@

from pathlib import Path

import pytest

ROOT = Path(__file__).resolve().parents[1]


def test_native_kernel_keeps_cuda_spelling_for_torchada() -> None:
if not (ROOT / ".git").exists():
pytest.skip("pre-build source contract requires a Git worktree")
kernel = (
ROOT
/ "csrc"
Expand Down
112 changes: 110 additions & 2 deletions tests/test_inplace_porting_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,82 @@

ROOT = Path(__file__).resolve().parent.parent

SUPPORTED_MUSA_STACKS = {
"torch==2.9.1.post1+musa5.2.0": {
"private": (
"torch_musa==2.9.1.post1+musa5.2.0",
"torchvision==0.24.1.post1+musa5.2.0",
"torchaudio==2.9.1+musa5.2.0",
"deep_ep==1.1.0+musa5.2.0torch2.9.1.post1s5000",
),
"torchada": "torchada==0.1.77",
},
"torch==2.11.0.post1+musa5.2.0": {
"private": (
"torch_musa==2.11.0.post1+musa5.2.0",
"torchvision==0.26.0.post1+musa5.2.0",
"torchaudio==2.11.0+musa5.2.0",
"deep_ep==1.1.0+musa5.2.0torch2.11.0.post1",
),
"torchada": "torchada==0.1.79",
},
}


def _select_musa_stack(private_requirements):
torch_pins = sorted(
line for line in private_requirements if line.startswith("torch==")
)
assert len(torch_pins) == 1, f"expected one torch pin, got {torch_pins}"
torch_pin = torch_pins[0]
assert torch_pin in SUPPORTED_MUSA_STACKS, f"unsupported MUSA stack: {torch_pin}"
return SUPPORTED_MUSA_STACKS[torch_pin]


def _declared_musa_stack():
private_requirements = set(
(ROOT / "requirements" / "musa_private.txt").read_text().splitlines()
)
common_requirements = set(
(ROOT / "requirements" / "common.txt").read_text().splitlines()
)
return (
private_requirements,
common_requirements,
_select_musa_stack(private_requirements),
)


def test_supported_musa_stack_contract_cases_are_explicit():
assert set(SUPPORTED_MUSA_STACKS) == {
"torch==2.9.1.post1+musa5.2.0",
"torch==2.11.0.post1+musa5.2.0",
}
assert (
SUPPORTED_MUSA_STACKS["torch==2.9.1.post1+musa5.2.0"]["torchada"]
== "torchada==0.1.77"
)
assert (
SUPPORTED_MUSA_STACKS["torch==2.11.0.post1+musa5.2.0"]["torchada"]
== "torchada==0.1.79"
)
assert (
"torchvision==0.24.1.post1+musa5.2.0"
in SUPPORTED_MUSA_STACKS["torch==2.9.1.post1+musa5.2.0"]["private"]
)
assert (
"torchvision==0.26.0.post1+musa5.2.0"
in SUPPORTED_MUSA_STACKS["torch==2.11.0.post1+musa5.2.0"]["private"]
)
for torch_pin, expected in SUPPORTED_MUSA_STACKS.items():
assert _select_musa_stack({torch_pin}) is expected


def test_torchada_floor_is_consistent():
assert 'dynamic = ["dependencies"]' in (ROOT / "pyproject.toml").read_text()
assert "torchada>=0.1.76" in (ROOT / "requirements" / "common.txt").read_text()
private_requirements, common_requirements, expected = _declared_musa_stack()
assert set(expected["private"]).issubset(private_requirements)
assert expected["torchada"] in common_requirements


def test_musa_image_runtime_dependency_contract():
Expand All @@ -22,8 +94,41 @@ def test_musa_image_runtime_dependency_contract():
dockerfile = (ROOT / "docker" / "musa.Dockerfile").read_text()

assert "triton==3.2.0" in private_requirements
_, _, expected = _declared_musa_stack()
assert set(expected["private"]).issubset(private_requirements)
assert "fastapi[standard]" in runtime_requirements
assert "pycountry" in runtime_requirements
assert '"requirements/common.txt"' in dockerfile
exact_import_gates = (
("torchada", "torchada"),
("torch", "torch"),
("torch_musa", "torch_musa"),
("torchvision", "torchvision"),
("torchaudio", "torchaudio"),
("deep_ep", "deep_ep"),
)
for dist_name, module_name in exact_import_gates:
gate = f'("{dist_name}", "{module_name}", requirement_prefix("{dist_name}"))'
assert gate in dockerfile

exact_version_gate = (
'exact_version_dists = frozenset({"torchada", "torch", "torch_musa", '
'"torchvision", "torchaudio", "deep_ep"})'
)
assert exact_version_gate in dockerfile
torchada_gate = '("torchada", "torchada", requirement_prefix("torchada"))'
torch_gate = '("torch", "torch", requirement_prefix("torch"))'
assert torchada_gate in dockerfile
assert dockerfile.index(torchada_gate) < dockerfile.index(torch_gate)
import_statement = "importlib.import_module(module_name)"
version_statement = "installed = version(dist_name)"
assert import_statement in dockerfile
assert dockerfile.index(import_statement) < dockerfile.index(version_statement)
assert "if dist_name in exact_version_dists and installed != prefix:" in dockerfile
assert (
"if dist_name not in exact_version_dists and prefix "
"and not installed.startswith(prefix):"
) in dockerfile
assert '("triton", "triton", requirement_prefix("triton"))' in dockerfile
assert '("uvloop", "uvloop", "")' in dockerfile
assert '("pycountry", "pycountry", "")' in dockerfile
Expand Down Expand Up @@ -275,7 +380,10 @@ def test_no_legacy_mirror_contract_remains():
"quantization_musa",
"per-file _musa",
)
paths = [ROOT / ".gitignore", ROOT / "third_party" / "PINS"]
paths = [ROOT / "third_party" / "PINS"]
gitignore = ROOT / ".gitignore"
if gitignore.exists():
paths.append(gitignore)
paths.extend(sorted((ROOT / "vllm_musa" / "patches" / "series").glob("*.patch")))

offenders = []
Expand Down
Loading