Repository navigation
Conversation
vllm 0.30.0's PyPI wheel is already a cu130 / torch-2.13 abi3 build, so it needs no wheelhouse routing and no source build, and it is the version the CUDA-13 environment is verified against (vllm 0.30.0.0+cu130 imports alongside sglang 0.5.20.0+cu130, verl 0.9.0.2+cu130 and torch 2.13.0.2+cu130 on B300). Also updates the adjacent comments: the flashinfer transitive pin moves to ==0.6.18.post1, and the transformers floor is stated for 0.30. OSS-facing change only: no internal build tags and no internal-only dependencies. The internal form of this bump is carried separately on arde/bump-vllm-0.30.
Completes the vllm 0.29.0 -> 0.30.0 bump by regenerating the lockfile, which was the outstanding WIP item on the internal branch. Note on scope: the lock moves more than vllm alone (nvidia-nccl-cu13 -> 2.30.7, triton -> 3.8.0, nvidia-cutlass-dsl -> 4.7.1, nvidia-cudnn-cu13 -> 9.24.0.43, quack-kernels -> 0.6.5, plus a torch 2.14.0 resolution for the non-extra markers). That is NOT an unrelated refresh: regenerating with `uv lock --upgrade-package vllm` produces the same set, because vllm 0.30.0 transitively requires those newer versions. Verified: `uv lock` resolves cleanly against PyPI.
|
|
arde171
changed the title
arde171
marked this pull request as ready for review
September 30, 2026 08:20
3 of 7 tasks
5 of 8 tasks
wuxibin89
added a commit
that referenced
this pull request
Oct 26, 2026
….16.1 (#8390) ### What does this PR do? Upgrade the CUDA 13 dependency stack to `vllm==0.32.0` and the project-wide `transformers==5.16.1` pin, then regenerate `uv.lock`. vLLM 0.32.0 requires `tokenizers>=0.23.2`, while SGLang 0.5.20 still declares `tokenizers==0.22.2`. Add a project-wide `tokenizers==0.23.2` override, following the existing Transformers override pattern, so every resolution fork remains consistent. Also widen the Transformers support window in `setup.py` and `requirements.txt` to `<5.17` so package metadata accepts the pinned version. Adapt `AsyncLLM.pause_generation()` to the vLLM 0.32 API while retaining vLLM 0.29 compatibility: 0.32 and newer use `mode="abort"`, while older versions keep `wait_for_inflight_requests=False`. Keep the score-centering top-k coverage test valid for both providers: vLLM 0.29 retains cutoff ties, while 0.32 keeps exactly `top_k` entries. Adapt the Qwen3.5 monkey patches to both Transformers 5.12 and 5.16. Transformers 5.16 renamed decoder `layer_type` to `block_type` and moved causal-convolution and gated-delta callables from instance attributes to module functions. This is not a duplicate of [#8072](#8072): that PR targets vLLM 0.30.0 only, whereas this PR targets 0.32.0 and includes the required Transformers 5.16.1, tokenizers 0.23.2, and compatibility changes. AI assistance was used to update dependency pins, regenerate the lockfile, diagnose CI, implement compatibility handling, run checks, and draft this PR. The submitter reviewed the changed files and confirmed the change. ### Checklist Before Starting - [x] Search for similar PRs. Query: https://github.com/verl-project/verl/pulls?q=is%3Apr+is%3Aopen+vllm+upgrade - [x] Format the PR title as `[{modules}] {type}: {description}` (This will be checked by the CI) ### Test - `uv lock` — resolved 378 packages successfully. - `uv lock --check` — passed. - `.venv-vllm/bin/python -m pytest tests/workers/rollout/rollout_vllm/test_submitter_gate_on_cpu.py -q` — 18 passed. - `.venv-vllm/bin/python -m pytest tests/workers/rollout/rollout_vllm/test_vllm_topk_support.py -q -k 'small_decode_batch_provider_support_coverage or pure_top_p_overflow'` — 5 passed. - `.venv-vllm/bin/torchrun --nproc_per_node=1 -m pytest tests/special_distributed/test_qwen35_linear_attention_ulysses_sp.py -q` — 2 passed. - Transformers 5.12.1 compatibility probe — legacy Qwen3.5 decoder and gated-delta instance attributes are detected. - `uvx --with hydra-core pre-commit run --all-files --show-diff-on-failure --color=always` — all hooks passed. - `uvx --with hydra-core pre-commit run --files verl/models/transformers/qwen3_5.py tests/workers/rollout/rollout_vllm/test_vllm_topk_support.py` — all hooks passed after the CI fixes. - `git diff --check` — passed. ### API and Usage Example No user-facing API change. Sync the vLLM environment through the existing extra: `uv sync --extra fsdp --extra vllm` ### Design & Code Changes - Pin vLLM 0.32.0 and Transformers 5.16.1 across the uv backend extras. - Override tokenizers at 0.23.2 to satisfy vLLM 0.32.0 and Transformers 5.16.1 while retaining SGLang 0.5.20. - Expand the package and development Transformers support window from `<5.13` to `<5.17`. - Select the legacy or current vLLM pause keyword according to `_VLLM_VERSION`, with regression coverage for 0.29.0 and 0.32.0. - Preserve conservative top-k support evidence across the different vLLM 0.29 and 0.32 cutoff-tie semantics. - Resolve Qwen3.5 decoder and kernel callables from their Transformers 5.12 instance attributes or Transformers 5.16 module-level replacements. - Regenerate the universal lockfile for all platform and backend resolution forks. ### Checklist Before Submitting - [x] Read the [Contribute Guide](https://github.com/verl-project/verl/blob/main/CONTRIBUTING.md). - [x] Apply [pre-commit checks](https://github.com/verl-project/verl/blob/main/CONTRIBUTING.md#code-linting-and-formatting): `pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always`. - [ ] Add / Update [the documentation](https://github.com/verl-project/verl/tree/main/docs). Not needed: this changes dependency pins and internal compatibility handling without changing user-facing APIs. - [x] Add unit or end-to-end test(s) to [the CI workflow](https://github.com/verl-project/verl/tree/main/.github/workflows) to cover all the code. Added pause API regression coverage and updated existing vLLM top-k and Qwen3.5 integration coverage. - [ ] Once your PR is ready for CI, send a message in [the `ci-request` channel](https://verl-project.slack.com/archives/C091TCESWB1) in [the `verl` Slack workspace](https://join.slack.com/t/verl-project/shared_invite/zt-3855yhg8g-CTkqXu~hKojPCmo7k_yXTQ). (If not accessible, please try [the Feishu group (飞书群)](https://applink.larkoffice.com/client/chat/chatter/add_by_link?link_token=772jd4f1-cd91-441e-a820-498c6614126a).) - [ ] If your PR is related to the `recipe` submodule, please also update the reference to the submodule commit via `git submodule update --remote` or `cd recipe && git pull origin main`. Not applicable. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Upgrade the rollout engine pin to the current release:
vllm==0.29.0->vllm==0.30.0, and regenerateuv.lockso the lock and the pin agree.This follows the same pattern as #7973 (
[rollout] chore: upgrade vllm==0.29.0/sglang==0.5.20), which performed the previous bump; this is the next step from 0.29.0.vllm 0.30.0's PyPI wheel is already a cu130 / torch-2.13 abi3 build, so no source build or wheelhouse routing is required.Checklist Before Starting
https://github.com/verl-project/verl/pulls?q=is%3Apr+is%3Amerged+vllm+upgrade
[{modules}] {type}: {description}(checked by CI){modules}=rollout;{type}=chore; no[BREAKING]Test
uv lockresolves cleanly against PyPI (exit 0), withvllmlocked at0.30.0.The lock also moves
nvidia-nccl-cu13 -> 2.30.7,triton -> 3.8.0,nvidia-cutlass-dsl -> 4.7.1,nvidia-cudnn-cu13 -> 9.24.0.43andquack-kernels -> 0.6.5. This is not an unrelated refresh: regenerating withuv lock --upgrade-package vllmproduces the same set, becausevllm 0.30.0transitively requires those newer versions.The GPU test jobs run on self-hosted runners that are not exposed to fork PRs (the job queues for 24h and is cancelled), so I have not been able to obtain a CI signal here. Happy to post in the
ci-requestchannel, or to run any specific command you want and paste the output.API and Usage Example
No API change: this is a dependency pin plus the regenerated lockfile.
Design & Code Changes
Two files:
pyproject.toml:vllm==0.29.0->vllm==0.30.0, plus the adjacent comments that name the version (the flashinfer transitive pin is==0.6.18.post1in 0.30; thetransformersfloor comment now refers to 0.30).uv.lock: regenerated.I deliberately did not touch
setup.py(vllm>=0.18.0, a floor that 0.30 already satisfies),requirements.txt(no vllm pin), ormanage_envs.py(no vllm version) — none of them need changing for this bump.I also have not ported any rollout code adaptation, because I have no evidence yet that 0.30 requires one: #7973's code changes (
vllm_rollout/weight_update_utils.py,models/mcore/patch.py,utils/vllm/utils.py) accompanied the torch 2.11 -> 2.13 bump as well. If you tell me 0.30 needs a specific adaptation, I will add it.Checklist Before Submitting
pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=alwaysci-requestchannel — pending, since this is currently a draftrecipesubmodule, please also update the reference to the submodule commit — not applicable