chore: Upgrade vLLM from 0.17.1 to 0.20.0 by kajalj22 · Pull Request #2384 · NVIDIA-NeMo/RL

kajalj22 · 2026-05-01T18:51:30Z

Summary

Upgrades vLLM from 0.17.1 to 0.20.0, along with dependent version bumps and API adaptation changes.

Dependency changes (`pyproject.toml`)

vLLM 0.17.1 → 0.20.0
torch 2.10.0 → 2.11.0
torchvision 0.25.0 → 0.26.0
flashinfer 0.6.4 → 0.6.8.post1
requires-python capped to <3.14

vLLM 0.20 API adaptation (`vllm_worker_async.py`)

vLLM 0.20 moved chat preprocessing from OpenAIServing._preprocess_chat to OpenAIServingRender.preprocess_chat
Created NeMoRLOpenAIServingRender(NeMoRLOpenAIServingMixin, OpenAIServingRender) to carry our required_prefix_token_ids override into the new render architecture
Removed mixin from Chat/Tokenization classes (they now delegate to the render object)
Added new required kwargs: reasoning_parser, skip_mm_cache, enable_auto_tools, openai_serving_render

Weight loading fix (`vllm_backend.py`)

Wrapped process_weights_after_loading calls in set_current_vllm_config context manager (required by vLLM 0.20)

Pickle fix (`megatron_policy_worker.py`)

Changed self.model.config → getattr(self.model, "config", None) to prevent cloudpickle from capturing torch.distributed.config (a non-pickleable ConfigModuleInstance in torch 2.11)

Test plan

CI unit tests pass (Generation, Policy, Other)
CI container build succeeds (including sglang)
Functional tests pass

🤖 Generated with Claude Code

copy-pr-bot · 2026-05-01T18:51:34Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

kajalj22 · 2026-05-01T19:36:40Z

/ok to test e2a1776

kajalj22 · 2026-05-06T19:57:51Z

/ok to test e52995e

kajalj22 · 2026-05-06T20:12:54Z

/ok to test 648be36

kajalj22 · 2026-05-13T03:52:13Z

/ok to test 21c0e8e

kajalj22 · 2026-05-13T20:16:18Z

/ok to test 711a112

copy-pr-bot · 2026-05-14T01:49:43Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

kajalj22 · 2026-05-14T21:22:03Z

/ok to test ba3ab1e

kajalj22 · 2026-05-20T22:41:53Z

/ok to test 197eb71

kajalj22 · 2026-05-21T04:35:59Z

/ok to test 97289f5

kajalj22 · 2026-05-21T04:47:23Z

/ok to test f19fc23

kajalj22 · 2026-05-21T15:25:44Z

/ok to test 132152f

kajalj22 · 2026-05-21T16:28:54Z

/ok to test fd73aa2

kajalj22 · 2026-05-21T23:18:29Z

/ok to test 1af50ac

- vLLM 0.17.1 → 0.20.0, torch 2.10 → 2.11, torchvision 0.25 → 0.26, flashinfer 0.6.4 → 0.6.8.post1 - Cap requires-python to <3.14 - Adapt to vLLM 0.20 render architecture: move prefix-token override to NeMoRLOpenAIServingRender - Wrap process_weights_after_loading in set_current_vllm_config context - Fix cloudpickle ConfigModuleInstance error (torch 2.11) - Fix Eagle3 draft weight loading: trim padded vocab embeddings Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>

vLLM 0.20 moved chat preprocessing to the render layer, but create_chat_completion no longer catches errors from that path. Prompts exceeding max_model_len now raise VLLMValidationError as an unhandled exception (500) instead of returning ErrorResponse (400). The Gym proxy only detects context-length overflow on 400, so the 500 crashes rollouts. Catch VLLMValidationError at the endpoint and return HTTP 400 to restore the graceful handling chain. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>

sglang is incompatible with the vLLM 0.20 / torch 2.11 upgrade. Unconditionally set SKIP_SGLANG_BUILD=1 so the Docker build and tests skip sglang until it is updated separately. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>

Eagle3 draft models can have draft_vocab_size (32k) different from vocab_size (151k). The old code used a single org_vocab_size from the first VocabParallelEmbedding (embed_tokens), which meant lm_head weights were never trimmed (padded_32k < 151k → condition false), causing an assertion failure in vLLM's weight_loader. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>

copy-pr-bot · 2026-05-21T23:20:25Z

Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

kajalj22 · 2026-05-21T23:22:26Z

/ok to test 92e00b7

vLLM 0.20 added quant_config arg to ParallelLMHead in Eagle3LlamaForCausalLM.__init__, so the old_snippet no longer matched and the has_own_lm_head patch silently failed to apply. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Kajal Jain <kajalj@nvidia.com>

kajalj22 · 2026-05-22T00:26:47Z

/ok to test a0f1d4b

kajalj22 · 2026-05-22T17:55:12Z

/ok to test 73e9b54

kajalj22 requested a review from a team as a code owner May 1, 2026 18:51

kajalj22 changed the title ~~Upgrade vLLM to 2.11~~ Upgrade vLLM to 0.20.0 May 1, 2026

kajalj22 marked this pull request as draft May 1, 2026 18:56

kajalj22 changed the title ~~Upgrade vLLM to 0.20.0~~ chore: Upgrade vLLM to 0.20.0 May 1, 2026

kajalj22 added the CI:L1 Run doctests, unit tests, and functional tests label May 1, 2026

copy-pr-bot Bot had a problem deploying to nemo-ci May 1, 2026 19:50 Failure

copy-pr-bot Bot had a problem deploying to nemo-ci May 6, 2026 19:58 Failure

kajalj22 force-pushed the kajalj/upgrade-vllm-2.11 branch from e52995e to f7ae9ae Compare May 6, 2026 20:00

copy-pr-bot Bot temporarily deployed to nemo-ci May 6, 2026 20:13 Inactive

copy-pr-bot Bot had a problem deploying to nemo-ci May 6, 2026 21:58 Failure

yuki-97 mentioned this pull request May 12, 2026

feat: Support Nemotron-nano-v3 Omni AutoModel Path #2362

Merged

copy-pr-bot Bot temporarily deployed to nemo-ci May 13, 2026 03:52 Inactive

copy-pr-bot Bot temporarily deployed to nemo-ci May 13, 2026 06:35 Inactive

kajalj22 force-pushed the kajalj/upgrade-vllm-2.11 branch from 21c0e8e to 711a112 Compare May 13, 2026 15:57

copy-pr-bot Bot had a problem deploying to nemo-ci May 13, 2026 20:16 Failure

copy-pr-bot Bot temporarily deployed to public May 13, 2026 20:16 Inactive

copy-pr-bot Bot temporarily deployed to public May 13, 2026 20:17 Inactive

copy-pr-bot Bot temporarily deployed to public May 13, 2026 20:20 Inactive

kajalj22 force-pushed the kajalj/upgrade-vllm-2.11 branch from 711a112 to a5f29ce Compare May 14, 2026 01:49

kajalj22 force-pushed the kajalj/upgrade-vllm-2.11 branch from a5f29ce to ba3ab1e Compare May 14, 2026 21:21

copy-pr-bot Bot temporarily deployed to public May 14, 2026 21:22 Inactive

kajalj22 mentioned this pull request May 20, 2026

ci: switch sglang to prebuilt PyPI wheels (v0.5.11) #2535

Merged

3 tasks

kajalj22 and others added 4 commits May 21, 2026 18:20

chtruong814 approved these changes May 22, 2026

View reviewed changes

bxyu-nvidia approved these changes May 22, 2026

View reviewed changes

ananthsub reviewed May 22, 2026

View reviewed changes

Comment thread pyproject.toml

ananthsub approved these changes May 22, 2026

View reviewed changes

Merge branch 'main' into kajalj/upgrade-vllm-2.11

73e9b54

sharonyu-115 mentioned this pull request May 29, 2026

feat: add Gemma4 support #2224

Merged

4 tasks

This was referenced May 29, 2026

Nightly test llm_dpo_qwen2_5_math7b_1n8g_megatron_chunked_linear_ce_loss failing metric check #2626

Closed

Nightly test failing lm_grpo_deepscaler_1_5b_16K and llm_grpo_deepscaler_1_5b_24K #2627

Closed

anwithk mentioned this pull request May 31, 2026

Bump vLLM version to 0.20.0 NVIDIA-NeMo/Gym#1475

Closed

sharonyu-115 mentioned this pull request Jun 2, 2026

fix: resolve qwen3.5-35ba3b megatron ep16 OOM via TP=2 (#2619) #2668

Closed

4 tasks

NolenLiang mentioned this pull request Jun 3, 2026

fix: lower DPO Qwen2.5-Math-7B accuracy threshold after torch 2.11 up… #2676

Merged

Uh oh!

Conversation

kajalj22 commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Dependency changes (pyproject.toml)

vLLM 0.20 API adaptation (vllm_worker_async.py)

Weight loading fix (vllm_backend.py)

Pickle fix (megatron_policy_worker.py)

Test plan

Uh oh!

copy-pr-bot Bot commented May 1, 2026

Uh oh!

kajalj22 commented May 1, 2026

Uh oh!

kajalj22 commented May 6, 2026

Uh oh!

kajalj22 commented May 6, 2026

Uh oh!

kajalj22 commented May 13, 2026

Uh oh!

kajalj22 commented May 13, 2026

Uh oh!

copy-pr-bot Bot commented May 14, 2026

Uh oh!

kajalj22 commented May 14, 2026

Uh oh!

kajalj22 commented May 20, 2026

Uh oh!

kajalj22 commented May 21, 2026

Uh oh!

kajalj22 commented May 21, 2026

Uh oh!

kajalj22 commented May 21, 2026

Uh oh!

kajalj22 commented May 21, 2026

Uh oh!

kajalj22 commented May 21, 2026

Uh oh!

copy-pr-bot Bot commented May 21, 2026

Uh oh!

kajalj22 commented May 21, 2026

Uh oh!

kajalj22 commented May 22, 2026

Uh oh!

Uh oh!

kajalj22 commented May 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kajalj22 commented May 1, 2026 •

edited

Loading

Dependency changes (`pyproject.toml`)

vLLM 0.20 API adaptation (`vllm_worker_async.py`)

Weight loading fix (`vllm_backend.py`)

Pickle fix (`megatron_policy_worker.py`)