Skip to content

Upgrade mlx-vlm and torchvision so Qwen3.5 multimodal will run#276

Merged
waybarrios merged 2 commits intowaybarrios:mainfrom
perry2of5:fix-qwen3.5-mllm-startup
Apr 11, 2026
Merged

Upgrade mlx-vlm and torchvision so Qwen3.5 multimodal will run#276
waybarrios merged 2 commits intowaybarrios:mainfrom
perry2of5:fix-qwen3.5-mllm-startup

Conversation

@perry2of5
Copy link
Copy Markdown
Contributor

@perry2of5 perry2of5 commented Apr 10, 2026

This PR upgrades libraries so --mllm will succeed. (Building for vision doesn't work on main right now.)

% git reset --hard origin/main
HEAD is now at b4fa030 Merge pull request #232 from sjswerdloff/fix/streaming-tool-call-content-leak
% rm -rf .venv uv.lock && uv venv --python 3.13 && source .venv/bin/activate && uv sync && uv pip install -e ".[dev,vision]"
Using CPython 3.13.11
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 291 packages in 2.29s
error: Distribution `torchvision==0.18.0 @ registry+https://pypi.org/simple` can't be installed because it doesn't have a source distribution or wheel for the current platform

hint: You're using CPython 3.13 (`cp313`), but `torchvision` (v0.18.0) only has wheels with the following Python ABI tags: `cp310`, `cp311`, `cp312`

I tested by running the following:

vllm-mlx serve --served-model-name default mlx-community/Qwen3.5-35B-A3B-4bit

I ran vllm-mlx-chat in another window and it successfully responded to "say hi" and "tell me how many tables are in the attached ERD".

Originally there was a note that this depends on PR 215 or PR 243 being applied first, but PR 268 fixed the issue so that is no longer relevant

@perry2of5
Copy link
Copy Markdown
Contributor Author

Updated to fix merge conflict. Works with

vllm-mlx serve --served-model-name default --mllm mlx-community/Qwen3.5-35B-A3B-4bit

and

vllm-mlx serve --served-model-name default mlx-community/Qwen3.5-35B-A3B-4bit

@waybarrios
Copy link
Copy Markdown
Owner

The torchvision bump makes sense, 0.18.0 has no cp313 wheels so this is needed for Python 3.13 users. One thing worth noting: torchvision 0.21.x requires torch 2.6.x, but the [vision] extras still declare torch>=2.3.0. pip will resolve it correctly through transitive deps, but bumping that floor to >=2.6.0 would make the constraint explicit. Up to you whether to include that in this PR or leave it for later.

Copy link
Copy Markdown
Owner

@waybarrios waybarrios left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me so far.

@waybarrios waybarrios merged commit 3110d1a into waybarrios:main Apr 11, 2026
7 checks passed
@perry2of5 perry2of5 deleted the fix-qwen3.5-mllm-startup branch April 11, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants