Skip to content

[LMI] Bump vLLM wheel to patched v0.28.0 and torch to 2.13.0 for LMI 29.0.0 - #3075

Open
shiehbeth wants to merge 1 commit into
masterfrom
lmi29-vllm-0.28.0-wheel
Open

shiehbeth wants to merge 1 commit into
masterfrom
lmi29-vllm-0.28.0-wheel

Conversation

@shiehbeth

@shiehbeth shiehbeth commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Description

Point the LMI container at the patched vLLM v0.28.0 wheel (base tag v0.28.0 2cf0a691 + 6 cherry-picked PRs: #44675, #48210, #48363, #48402, #50040, #51742 for the DJL 0.37.0 / LMI 29.0.0 release.

Wheel: https://publish.djl.ai/vllm/vllm-0.28.0-cp312-cp312-linux_x86_64.whl
sha256 100e71ec575de176e717fb396f97a5e3cb207b41d5457117b8690ca2fa55a9b3, built py3.12 / CUDA 13.0 (cu130), TORCH_CUDA_ARCH_LIST=8.0;8.6;8.9;9.0;10.0;12.0.

torch 2.11.0 → 2.13.0 is required, not optional. vLLM 0.28.0 pins torch==2.13.0 (requirements/cuda.txt), and the wheel's compiled .so files are torch-ABI-locked — so the container runtime must move together with the wheel. torchvision moves 0.26.0 → 0.28.0 for the same reason. This is the one difference in shape from the LMI 28.0.0 bump, which was wheel-URL only.

Two notes for reviewers:

  • torchaudio stays at 2.11.0 and is deliberately not pinned here. 2.11.0 is the newest released torchaudio — the torch/torchaudio version numbers have decoupled this cycle, so torchaudio==2.13.0 does not exist. This file pins neither torchaudio nor torchcodec, so both resolve transitively; all four (torch 2.13.0+cu130, torchvision 0.28.0+cu130, torchaudio 2.11.0+cu130, torchcodec 0.14+) are confirmed present on the cu130 index.
  • The pyproject.toml [test] pin bump is not cosmetic. A stale vllm== pin there breaks test_vllm_embedding with module 'djl_python.lmi_vllm' has no attribute 'vllm_async_service'.

--extra-index-url .../whl/cu130 is unchanged, and lmi.Dockerfile needs no change (already nvidia/cuda:13.0.2-devel-ubuntu24.04 / cuda_version=cu130 / python_version=3.12).

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Checklist:

Feature/Issue validation/testing

Pre-merge verification completed against the built artifact (not predicted):

  • Wheel published and reachable. https://publish.djl.ai/vllm/vllm-0.28.0-cp312-cp312-linux_x86_64.whl returns HTTP 200, content-length: 625092004. The object was read back out of S3 and re-hashed to 100e71ec…, matching the build byte-for-byte.
  • Pin bumps taken from the wheel's own METADATA, not inferred: Requires-Dist: torch==2.13.0, torchaudio==2.11.0, torchvision==0.28.0, torchcodec>=0.14.
  • Dependency pin cross-check clean. All 5 pins overlapping between lmi-container-requirements.txt and vLLM 0.28.0's requirements/{common,cuda}.txt agree — torch/torchvision now match exactly, transformers >= 5.5.3 and compressed-tensors == 0.17.0 were already byte-identical, huggingface-hub resolves. 0 conflicts.
  • djl_python ↔ vLLM 0.28.0 API compatibility check clean (static, AST). 35/35 distinct vllm.* import statements resolve (module found and imported symbol present), and 4/4 constructor call sites accept every kwarg passed against the real __init__ signatures (OnlineRenderer, OpenAIServingChat, OpenAIServingCompletion, ServingEmbedding). No glue adaptation appears to be needed for this bump.
  • GPU arch coverage confirmed from the binary with cuobjdump: _C_stable_libtorch.abi3.so and _moe_C_stable_libtorch.abi3.so both carry compiled SASS for sm_80, sm_86, sm_89, sm_90, sm_90a, sm_100 and sm_120.

Full container integration testing runs on the nightly build that this merge enables.

…29.0.0

Point the LMI container at the patched vLLM v0.28.0 wheel for the
DJL 0.37.0 / LMI 29.0.0 release.

torch 2.11.0 -> 2.13.0 is required, not optional: vLLM 0.28.0 pins
torch==2.13.0, and the wheel's compiled .so files are torch-ABI-locked,
so the container runtime must move together with the wheel.

torchaudio stays at 2.11.0 (newest released; the torch/torchaudio
version numbers have decoupled this cycle) and resolves transitively.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@xyang16
xyang16 force-pushed the lmi29-vllm-0.28.0-wheel branch from 4054f7b to 5d65f4a Compare September 11, 2026 05:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant