[faresobeid]

Bumps vLLM 0.31.0 -> 0.32.0.

Dependencies

• vllm 0.32.0 wheel. torch stays 2.13.0, flashinfer stays 0.7.0.post1.
• transformers 5.6.2 -> 5.19.0 (vLLM 0.32 requires >=5.16.1,<5.20; 5.19.0 is what vLLM's CI tests). The transformers==5.6.2 override is removed. tokenizers 0.22.2 -> 0.23.3.
• nvidia-cutlass-dsl override 4.7.1 -> 4.8.0 (exact pin in vLLM 0.32). tokenspeed-mla 0.2.15, humming-kernels 0.1.17, instanttensor 0.2.2, xgrammar 0.2.8 come from vLLM; humming-kernels and instanttensor are exempt from the 7-day cutoff.
• ifbench is excluded from the workspace: it pins setuptools<78, vLLM 0.32 needs >=78.1.1. No config or test references it.

Patches deleted (fixed upstream in 0.32)

• monkey_patch_shm_buffer_ownership: [Bugfix][Distributed] Own SHM out-of-band buffers before releasing read slots vllm-project/vllm#61000.
• monkey_patch_tokenize_params_validation: [Bugfix] Treat max_tokens as output upper bound, not input-space reservation vllm-project/vllm#57035 (max_tokens is an upper bound, not reserved from the context).
• monkey_patch_nano_v3_reasoning_parser: upstream nemotron_v3 parser. The Nemotron 3 Nano / 3.5 auto-detection now picks nemotron_v3. Configs that set reasoning_parser = "nano_v3" must rename it.
• DeepSeek V4 layer_types shim (utils/transformers_compat.py and its vLLM patch): transformers >= 5.15 knows the V4 layer types.

Breakages fixed

• serving_tokens.py: GenerateResponseChoice is gone and GenerateResponse is now a discriminated union ([Frontend] Add output_mode to /inference/v1/generate (RFC #56851 Phase 1) vllm-project/vllm#58588). Subclass GenerateTokensChoice / GenerateTokensResponse and check isinstance against GenerateTokensResponse.
• prefill_logprobs: validate with GenerateTokensResponse (the union alias has no model_validate_json).
• _patch_lora_key_prefix: re-copied from 0.32's LoRAModel.from_local_checkpoint; tensorizer support was removed upstream ([LoRA] Remove tensorizer vllm-project/vllm#60024).
• NemotronH config: transformers 5.19 renames legacy layer types (mamba → linear_attention, attention → full_attention) by setting config.layer_types, which was a read-only property. layer_types is now an attribute_map alias of layers_block_type (as upstream), and the model and conversion code use the new names. Hub configs with legacy layers_block_type or hybrid_override_pattern load as before.
• Qwen3.5 tests: from_config through the auto class now swaps a composite config for its text config; the tests build VLMs with Qwen3_5ForCausalLM, as the trainer does.
• monkey_patch_kv_xfer_finished_tolerate_freed: synced the copied method with 0.32 (_set_kv_fetch_stage(req, KV_FETCH_COMPLETED_WAITING)).

Still needed on 0.32 (fixes land in 0.33 or are not merged): DeepSeek V4 C128 boundary, DeepSeek V4 attn_sink loading, GPT-OSS weight loading, KV-transfer freed assert, MiniMax think-end, routed experts with P/D connectors. Every remaining patch target still exists in 0.32; targets whose source changed were checked against the new source.

Validation

CPU: pytest tests/unit -m "not gpu" 854 passed; ruff clean. Import/attribute check of every vllm import and monkey-patch target in src/ and tests/ against 0.32: no missing names; patched methods whose source changed were reviewed.

GPU (H200, max 2 nodes; 0.31 baselines use main's lock at the same base commit, same nodes):

Check	0.32	0.31
tests/unit -m gpu (NemotronH, Qwen3.5, Mamba CP, DSv4 kernels, …)	pass after the fixes above¹	–
Integration: reverse_text, reverse_text_moe, reverse_text_lora, reverse_text_rl_opd, reverse_text_rl_sft, reverse_text_sft, reverse_text_sft_lora, alphabet_sort	pass²	–
Qwen3-30B-A3B RL, bf16, NCCL (15 steps, median step / mismatch KL)	15.3 s / 0.028	14.9 s / 0.025
Qwen3-30B-A3B RL, online fp8 (fp8_per_block), NCCL	15.4 s / 0.061	16.4 s / 0.063
Qwen3-30B-A3B RL, bf16, filesystem broadcast (trainer step)	156 s / 0.030	155 s / 0.020
GLM-4.5-Air RL, router replay, 1 train + 1 infer node (10 steps, median step / mismatch KL / reward)	23.4 s / 0.0028 / 0.49→0.65	24.7 s / 0.0036 / 0.44→0.66
Qwen3-30B-A3B P/D (NIXL 1P1D, vllm-router 0.2.3) vs colocated, router replay + sampling masks, batch-invariant: identical tokens, rows, masks, logprobs	c1 10/10, c10 20/20, uint16 c1 10/10 / c10 20/20, decode preemption 20/20	same as 0.31 results
GLM-4.5-Air (TP4) P/D vs prefill server queried directly (same node), batch-invariant: identical tokens and rows	0/10 (decode diverges); 10/10 with disable_custom_all_reduce = true³	10/10
Long multi-turn prompt (14.4k prompt + max_tokens 4096, max_model_len 16384), plain and router-replay server	200, prompt not truncated; /generate 15.9k + 4096 → 200, capped at max_model_len; 16.4k prompt → 400	same
DeepSeek-V4-Flash TP8+EP serving smoke	loads, coherent completion	–

¹ The DSv4 cudnn_flashmla kernel tests fail the same way on main when vLLM's DSv4 test loads first (FlashMLA symbol clash); with #4139 applied they pass on 0.32 (123 passed).
² Each integration module's test_dashboard needs playwright install --with-deps, which fails on our nodes; all other tests pass.

³ 0.32 uses the custom all-reduce under VLLM_BATCH_INVARIANT=1 (vllm-project/vllm#58623). Two TP>1 servers then no longer decode bit-identically, so P/D only matches colocated serving bit-for-bit with disable_custom_all_reduce = true. prime-rl does not set batch invariance; RL is unaffected (GLM row above).

Not covered: Qwen3.5/Qwen3.8 PLE reload (no checkpoint on disk); GLM-4.5-Air disaggregated RL (needs 3 nodes). Router-replay requests need top_k > 0 and temperature > 0 (sampling-mask replay), on 0.31 and 0.32 alike.

---

Note

Medium Risk
Touches core inference dependencies and removes several runtime patches; misalignment with vLLM 0.32 APIs or Nemotron parser changes could break rollouts or serving until configs are updated.

Overview
Upgrades the inference stack from vLLM 0.31.0 to 0.32.0, with matching pins: transformers 5.19.0, nvidia-cutlass-dsl 4.8.0, and direct wheel URLs. ifbench is dropped from the uv workspace because its setuptools<78 pin conflicts with vLLM 0.32.

Several vLLM workarounds are removed now that upstream fixed them: SHM buffer ownership on dequeue, relaxed TokenizeParams prompt-length checks, the custom nano_v3 reasoning parser, and the DeepSeek V4 layer_types transformers shim (transformers_compat.py). Nemotron 3 Nano / 3.5 auto-detection now uses the built-in nemotron_v3 parser instead of nano_v3.

API follow-ups for 0.32: token-in/token-out serving and prefill_logprobs use GenerateTokensChoice / GenerateTokensResponse instead of the old GenerateResponse* types. The LoRA checkpoint patch is re-synced from 0.32 (tensorizer path removed). The KV-transfer “freed request” patch adds _set_kv_fetch_stage(..., KV_FETCH_COMPLETED_WAITING) to match upstream.

Trainer Nemotron-H aligns layer-type names with newer transformers (linear_attention, full_attention) via attribute_map, while still accepting legacy mamba / attention in layers_block_type.

^{Reviewed by Cursor Bugbot for commit f92627e. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f92627e. Configure here.}

[cursor]

Legacy layer names stay unmapped

Medium Severity

The new legacy_config assertion expects mamba and attention to become linear_attention and full_attention, but NemotronHConfig stores layers_block_type unchanged and never remaps. layer_types is only an attribute_map alias, so this check fails or construction raises when transformers validates the old names against the new vocabulary. This GPU-marked test was not in the non-GPU run, so the gap was not caught.

 

^{Reviewed by Cursor Bugbot for commit f92627e. Configure here.}