Skip to content

Upgrade torch to 2.12.1 and torchvision to 0.27.1 - #3311

Open
QiliangCui wants to merge 1 commit into
mainfrom
cuiq-torch-213
Open

QiliangCui wants to merge 1 commit into
mainfrom
cuiq-torch-213

Conversation

@QiliangCui

@QiliangCui QiliangCui commented Jul 31, 2026 •

Copy link
Copy Markdown
Collaborator

Purpose

vLLM main moved to torch 2.13.0 in vllm-project/vllm#48155 (Jul 23), and the current vLLM LKG is well past that commit. tpu-inference has no direct torch pin — the torchvision==0.25.0 pin implicitly kept the TPU image on torch 2.10.0+cpu, so we have been drifting from the torch version vLLM develops and tests against.

torch 2.13 is currently blocked by torchax: torch 2.13 removed the aten.prod.dim_Dimname overload (torch.ops.aten.prod.overloads() = ['default', 'dim_int', 'int_out', 'out']), and torchax references it at import time in amp.py — so import torchax raises AttributeError under torch 2.13. This affects every released torchax version including the current 0.0.14 nightlies, confirmed both in CI (every test importing torchax failed at collection) and in a local venv repro.

torch 2.12.1 retains the Dimname overloads and torchax==0.0.13 imports cleanly under it. So this PR moves to torch 2.12.1 / torchvision 0.27.1 (torchvision hard-pins its paired torch, so the two move in lockstep), closing most of the gap. 2.13 can follow once torchax adapts.

Test Plan

  • Pre-commit CI on this PR.
  • Full nightly suites (NIGHTLY=1, plain + MODEL_IMPL_TYPE=vllm) run on this branch over the weekend (Jul 31 – Aug 2) via the dev pipeline, failure sets diffed against the same nights' main nightlies.

Test Result

Zero torch-attributable failures across 7 nightly runs + 2 pre-commit builds.

  • Pre-commit: build 23626 passed; build 23652 (after rebase onto current main) passed (2 one-off agent infra failures, green on retry).
  • Plain nightly: dev builds 720, 728, 733, 762 — every failure was also failing on the same night's main nightly with an identical signature (e.g. is_stale kwarg drift and the MM fault_tolerance_config issue, both since fixed on main by Fix nightly failures against new vLLM LKG (is_stale kwarg + MM test fault_tolerance_config) #3312; tpu6e Perf TP; DeepSeek-V4-Flash torchax MMLU). Aug 1–2 runs after rebasing onto main: failure set matched main's exactly.
  • MODEL_IMPL_TYPE=vllm nightly (the torchax-heavy path): dev builds 729, 734, 763 — same result; Aug 2 run's 11 failures were all shared with main's same-night vllm nightly. One TPU initialization GRPC_ERROR Ray-worker flake passed on retry.
  • torch 2.13.0 attempt for reference: builds 23611 / dev 715 — all torchax-importing tests failed at collection with the dim_Dimname AttributeError, which led to the torchax finding above.

@github-actions

Copy link
Copy Markdown

Description

Start with a short description of what the PR does and how this is a change from
the past.

The rest of the description includes relevant details and context, examples:

  • why is this change being made,
  • the problem being solved and any relevant context,
  • why this is a good solution,
  • some information about the specific implementation,
  • shortcomings of the solution and possible future improvements.

If the change fixes a Github issue, please include a link, e.g.,:
FIXES: #123456

Tests

Please describe how you tested this change, and include any instructions and/or
commands to reproduce.

Checklist

Before submitting this PR, please make sure:

  • I have performed a self-review of my code.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have made or will make corresponding changes to any relevant documentation.

@QiliangCui QiliangCui added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 31, 2026
vLLM main moved off torch 2.11 (currently 2.13.0 since
vllm-project/vllm#48155), while tpu-inference has
no direct torch pin and the torchvision==0.25.0 pin kept the TPU image on
torch 2.10.0+cpu. torch 2.13 is currently blocked: torchax (all released
versions incl. 0.0.14 nightlies) fails at import because torch 2.13
removed the aten.prod.dim_Dimname overload referenced in torchax amp.py.
torch 2.12.1 retains the Dimname overloads and torchax 0.0.13 imports
cleanly, so move to 2.12.1 now and revisit 2.13 when torchax adapts.
torchvision must move in lockstep (0.27.1 hard-pins torch==2.12.1).

Signed-off-by: Qiliang Cui <cuiq@google.com>
@QiliangCui
QiliangCui marked this pull request as ready for review August 2, 2026 11:54
@QiliangCui QiliangCui changed the title Upgrade torch to 2.13.0 and torchvision to 0.28.0 Upgrade torch to 2.12.1 and torchvision to 0.27.1 Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant