Skip to content

[PyTorch-MPS] Pin nightly Torch for earlier GPU submission #1117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
WindChimeRan wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[PyTorch-MPS] Pin nightly Torch for earlier GPU submission #1117

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c1d27dc
Pin PyTorch nightly for earlier MPS submission
WindChimeRan Oct 15, 2026
93ae0bc
Preserve wheel metadata through Homebrew cleanup
WindChimeRan Oct 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 6 additions & 10 deletions Formula/vllm-metal.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,25 +37,21 @@ def install
system libexec/"bin/pip", "install", vllm_wheel,
buildpath/"vllm_metal-#{version}-cp312-cp312-macosx_15_0_arm64.whl"

# TVM-FFI uses wheel RECORD files to locate its own and xgrammar's native
# libraries. Save them outside site-packages before Homebrew cleans them.
site_packages = libexec/"lib/python3.12/site-packages"
torch_requirements = site_packages/"vllm_metal/requirements-torch.txt"
system libexec/"bin/pip", "install", "-r", torch_requirements if torch_requirements.exist?

# TVM-FFI uses wheel RECORD files to locate its own and xgrammar's native
# libraries. Keep the files in pkgshare and link them back so cleanup preserves them.
%w[apache_tvm_ffi xgrammar].each do |package|
record = site_packages.glob("#{package}-*.dist-info/RECORD").fetch(0)
(pkgshare/"#{package}.record").write record.read
record.dirname.install_symlink pkgshare/"#{package}.record" => "RECORD"
end

bin.install_symlink libexec/"bin/vllm"
end

def post_install
site_packages = libexec/"lib/python3.12/site-packages"
%w[apache_tvm_ffi xgrammar].each do |package|
metadata = site_packages.glob("#{package}-*.dist-info").fetch(0)
cp pkgshare/"#{package}.record", metadata/"RECORD"
end
end

def caveats
<<~EOS
Start a server with:
Expand Down
8 changes: 8 additions & 0 deletions install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,13 @@ download_and_install_wheel() {
exit 1
fi

# Use this wheel's Torch pins. Older releases retain vLLM's defaults.
local torch_requirements
torch_requirements=$(python -I -c 'from importlib.metadata import distribution; print(distribution("vllm-metal").locate_file("vllm_metal/requirements-torch.txt"))')
if [[ -f "$torch_requirements" ]]; then
uv pip install -r "$torch_requirements"

@LxYuan0420 LxYuan0420 Oct 18, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we keep vLLM 0.31’s Torch 2.13 default and make this nightly stack an explicit PyTorch-MPS opt-in? My concern is that it overrides the supported stack for every install, while the measured gain is MPS-only.

@WindChimeRan WindChimeRan Oct 19, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also don't want to over-complicates UX - Users should not be expected to memorize or search for how to install these correctly. homebrew should be the default setting for most users.

Right now the pytorch-mps is still experimental. I plan to mark it as "usable" once there is one feature better than mlx path (such as the Qwen3-TTS, waiting for upstream merge now). So I marked this PR as draft, it's a bit overkill for now without real benefits.

Eventually I want the user just homebrew install, then they can run all features. currently tts and gguf are not in the homebrew path, so I opened #1161 .

After upstream omni PR merged, I'll run more tests on if we can bring pytorch-mps, nightly wheel, omni to the default installation (if the dependency cost is small, and the performance & UX gain are big)

@WindChimeRan WindChimeRan Oct 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MLX path throughput is unchanged, mainly because it's in low coupling with pytorch (we only use it for scheduler, meta data bookkeeping and some sampling fallbacks):

Concurrency Torch 2.13 Pinned nightly Change
1 150.1 tokens/s 150.6 tokens/s +0.3%
8 552.5 tokens/s 550.9 tokens/s −0.3%

Qwen3-0.6B, 100 requests per setting, 1,024 input/128 output tokens.

fi

success "Installed ${package_name}"
}

Expand Down Expand Up @@ -272,6 +279,7 @@ EOF

# Source checkouts build native artifacts; release installs use the wheel.
uv pip install -e ".[build]"
uv pip install -r vllm_metal/requirements-torch.txt
ensure_metal_toolchain
build_native_artifacts
else
Expand Down
1 change: 1 addition & 0 deletions scripts/lib.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ verify_wheel_artifacts() {
local expected
if ! expected=$(python -c "
from vllm_metal.metal.build import METALLIB_NAMES, NAX_METALLIB_NAME, metallib_path, mlx_version_path, output_path, mps_output_path, mps_version_path
print('requirements-torch.txt')
print(output_path().name)
print(mlx_version_path().name)
print(mps_output_path().name)
Expand Down
1 change: 1 addition & 0 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ def has_ext_modules(self):
setup(
distclass=BinaryDistribution,
package_data={
"vllm_metal": ["requirements-torch.txt"],
"vllm_metal.pytorch_backend": [
"*.mm",
f"_mps_ops{sysconfig.get_config_var('EXT_SUFFIX')}",
Expand Down
6 changes: 6 additions & 0 deletions vllm_metal/requirements-torch.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Override vLLM's Torch pin for upstream MPS submission improvements:
# https://github.com/pytorch/pytorch/pull/200181
--extra-index-url https://download.pytorch.org/whl/nightly/cpu
torch==2.16.0.dev20261013
torchvision==0.30.0.dev20261013
torchaudio==2.11.0.dev20261013
Loading