Upgrade torch to 2.12.1 and torchvision to 0.27.1 - #3311
Open
QiliangCui wants to merge 1 commit into
Open
QiliangCui wants to merge 1 commit into
QiliangCui wants to merge 1 commit into
Conversation
DescriptionStart with a short description of what the PR does and how this is a change from The rest of the description includes relevant details and context, examples:
If the change fixes a Github issue, please include a link, e.g.,: TestsPlease describe how you tested this change, and include any instructions and/or ChecklistBefore submitting this PR, please make sure:
|
QiliangCui
force-pushed
the
cuiq-torch-213
branch
from
July 31, 2026 22:08
7fb8417 to
e54a6c4
Compare
QiliangCui
force-pushed
the
cuiq-torch-213
branch
from
July 31, 2026 23:51
e54a6c4 to
223a78e
Compare
vLLM main moved off torch 2.11 (currently 2.13.0 since vllm-project/vllm#48155), while tpu-inference has no direct torch pin and the torchvision==0.25.0 pin kept the TPU image on torch 2.10.0+cpu. torch 2.13 is currently blocked: torchax (all released versions incl. 0.0.14 nightlies) fails at import because torch 2.13 removed the aten.prod.dim_Dimname overload referenced in torchax amp.py. torch 2.12.1 retains the Dimname overloads and torchax 0.0.13 imports cleanly, so move to 2.12.1 now and revisit 2.13 when torchax adapts. torchvision must move in lockstep (0.27.1 hard-pins torch==2.12.1). Signed-off-by: Qiliang Cui <cuiq@google.com>
QiliangCui
force-pushed
the
cuiq-torch-213
branch
from
August 1, 2026 13:26
223a78e to
a60ec1e
Compare
QiliangCui
marked this pull request as ready for review
August 2, 2026 11:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
vLLM main moved to torch 2.13.0 in vllm-project/vllm#48155 (Jul 23), and the current vLLM LKG is well past that commit. tpu-inference has no direct torch pin — the
torchvision==0.25.0pin implicitly kept the TPU image on torch 2.10.0+cpu, so we have been drifting from the torch version vLLM develops and tests against.torch 2.13 is currently blocked by torchax: torch 2.13 removed the
aten.prod.dim_Dimnameoverload (torch.ops.aten.prod.overloads()=['default', 'dim_int', 'int_out', 'out']), and torchax references it at import time inamp.py— soimport torchaxraisesAttributeErrorunder torch 2.13. This affects every released torchax version including the current 0.0.14 nightlies, confirmed both in CI (every test importing torchax failed at collection) and in a local venv repro.torch 2.12.1 retains the Dimname overloads and
torchax==0.0.13imports cleanly under it. So this PR moves to torch 2.12.1 / torchvision 0.27.1 (torchvision hard-pins its paired torch, so the two move in lockstep), closing most of the gap. 2.13 can follow once torchax adapts.Test Plan
NIGHTLY=1, plain +MODEL_IMPL_TYPE=vllm) run on this branch over the weekend (Jul 31 – Aug 2) via the dev pipeline, failure sets diffed against the same nights' main nightlies.Test Result
Zero torch-attributable failures across 7 nightly runs + 2 pre-commit builds.
is_stalekwarg drift and the MMfault_tolerance_configissue, both since fixed on main by Fix nightly failures against new vLLM LKG (is_stale kwarg + MM test fault_tolerance_config) #3312;tpu6e Perf TP;DeepSeek-V4-Flash torchax MMLU). Aug 1–2 runs after rebasing onto main: failure set matched main's exactly.MODEL_IMPL_TYPE=vllmnightly (the torchax-heavy path): dev builds 729, 734, 763 — same result; Aug 2 run's 11 failures were all shared with main's same-night vllm nightly. OneTPU initialization GRPC_ERRORRay-worker flake passed on retry.dim_DimnameAttributeError, which led to the torchax finding above.