Merge upstream vLLM main by hynky1999 · Pull Request #5 · macrodata-labs/vllm

Hynek Kydlíček (hynky1999) · 2026-06-02T13:07:36Z

Summary

Merge current vllm-project/vllm:main into macrodata-labs/vllm:main.

The only manual conflict resolution was in vllm/entrypoints/chat_utils.py, where the fork's "video" parser mapping and upstream's "tool_reference" parser mapping were both preserved.

Validation

merge-conflict pre-commit hooks run during commit:
- ruff check
- ruff format
- typos
- mypy
- SPDX/root import/forbidden import/config checks
git diff --check

AI assistance was used to prepare this merge.

…on (vllm-project#42787) Signed-off-by: Hua Huang <huah@nvidia.com> Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

Signed-off-by: holegots <ikun3.1415927@gmail.com>

) Signed-off-by: holegots <ikun3.1415927@gmail.com>

Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com>

…oject#43392)

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>

…ject#42691) Signed-off-by: sfeng33 <4florafeng@gmail.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

…ct#43494) Signed-off-by: Dao Le <daole@inferact.ai> Signed-off-by: Dao Le <Dao007forever@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>

Signed-off-by: Or Ozeri <oro@il.ibm.com>

…llm-project#43016) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>

…3083) Signed-off-by: Banani Ghosh <bg2502@nyu.edu> Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com> Co-authored-by: Banani Ghosh <bg2502@nyu.edu>

Signed-off-by: Rotem Shavitt <rshavitt@gmail.com> Signed-off-by: Or Ozeri <oro@il.ibm.com> Co-authored-by: Or Ozeri <oro@il.ibm.com>

Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>

…project#42373) Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…root target (vllm-project#40275) Signed-off-by: TheDuyIT <nduy250299@gmail.com> Signed-off-by: dtnguyen <dtnguyen@nvidia.com> Co-authored-by: Claude <noreply@anthropic.com>

…ject#42296) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>

…3568) Signed-off-by: esmeetu <jasonailu87@gmail.com>

Signed-off-by: Yihuki <wangbovbvb@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

…43281) Signed-off-by: Zhewen Li <zhewenli@inferact.ai> Co-authored-by: Zhewen Li <zhewenli@inferact.ai>

…t#43583) Signed-off-by: NickLucche <nlucches@redhat.com>

Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>

…mpler (vllm-project#43028) Signed-off-by: chaojun-zhang <chaojun.zhang@intel.com> Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

…ect#43632) Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

…in load_mask (vllm-project#43516) Signed-off-by: Dao Le <daole@inferact.ai> Signed-off-by: Dao Le <Dao007forever@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>

) Signed-off-by: Dao Le <Dao007forever@gmail.com>

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

Signed-off-by: Will.hou <1205157517@qq.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

…model. (vllm-project#43770) Signed-off-by: zzt <zengzetang.zzt@antgroup.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>

… fixture (vllm-project#44146) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>

…erformance improvement (vllm-project#43706) Signed-off-by: yewentao256 <zhyanwentao@126.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

…llm-project#43992) Signed-off-by: Madeesh Kannan <madeeswaran.kannan@jetbrains.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

…4161) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>

) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Micah Williamson <micah.williamson@amd.com> Co-authored-by: Micah Williamson <micah.williamson@amd.com>

…3779) Signed-off-by: xunzhuo <xunzhuo@vllm-semantic-router.ai> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

… updates (vllm-project#40096) Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com>

Signed-off-by: Nick Hill <nickhill123@gmail.com>

…ject#44234) Signed-off-by: Nick Hill <nickhill123@gmail.com>

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

…-project#44256) Signed-off-by: JartX <sagformas@epdcenter.es>

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

…nnector (vllm-project#43742) Signed-off-by: Dao Le <Dao007forever@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>

Signed-off-by: Micah Williamson <micah.williamson@amd.com>

…ct#44266) Signed-off-by: Alec Flowers <aflowers@nvidia.com>

vllm-project#44017) Signed-off-by: sfeng33 <4florafeng@gmail.com>

…llm-project#44131) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

…ject#38053) Signed-off-by: Krishna Chaitanya Balusu <krishnabkc15@gmail.com> Signed-off-by: wjinxu <1299461899@qq.com> Signed-off-by: Kc Balusu <kcbalusu@users.noreply.github.com> Co-authored-by: wjinxu <1299461899@qq.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> Co-authored-by: Kc Balusu <kcbalusu@users.noreply.github.com>

…llm-project#42959) Signed-off-by: Or Ozeri <oro@il.ibm.com>

…vllm-project#43930) Signed-off-by: Chaojun,Zhang <chaojun.zhang@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>

Signed-off-by: JooHo Lee <jooho414@gmail.com>

…m-project#43798) Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> Co-authored-by: ZiTian Zhao <zitian.zhao@tencentmusic.com> Co-authored-by: B-201 <Joy25810@foxmail.com>

…rid + fp8 KVCache (vllm-project#43990) Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>

…m-project#43991) Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>

…llm-project#44267) Signed-off-by: sfeng33 <4florafeng@gmail.com>

# Conflicts: # vllm/entrypoints/chat_utils.py Signed-off-by: Hynek Kydlicek <kydlicek.hynek@gmail.com>

Hua Huang (huanghua1994) and others added 30 commits May 23, 2026 16:08

[MM] Enable FlashInfer metadata support for Qwen2.5-VL vision attenti…

a0be71e

…on (vllm-project#42787) Signed-off-by: Hua Huang <huah@nvidia.com> Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>

[Docs] Fix stale version number in token_embed.md (vllm-project#43488)

7c2ff1f

Signed-off-by: holegots <ikun3.1415927@gmail.com>

[Docs] Fix stale version number in token_classify.md (vllm-project#43489

8737e4a

) Signed-off-by: holegots <ikun3.1415927@gmail.com>

[MoE] Migrate W4A8 CT to oracle kernel setup (vllm-project#42680)

4438b6e

Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com>

[Mooncake] Add metrics for MooncakeStoreConnector operations (vllm-pr…

819c610

…oject#43392)

[ROCm][Critical] Fix the GDN import bug (vllm-project#43486)

46f95b2

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>

Revert "[Misc] add humming to dependencies" (vllm-project#43492)

10d264a

[Bugfix] Fix reasoning dropped on streaming boundary deltas (vllm-pro…

b32fe41

…ject#42691) Signed-off-by: sfeng33 <4florafeng@gmail.com>

[Model Runner v2] Force v1 runner for tests (vllm-project#43233)

33d7cbe

Signed-off-by: yewentao256 <zhyanwentao@126.com>

[KV Connector] Keep MooncakeStore full hits block-aligned (vllm-proje…

0902d8e

…ct#43494) Signed-off-by: Dao Le <daole@inferact.ai> Signed-off-by: Dao Le <Dao007forever@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>

[kv_offload]: Add DSv4 support (vllm-project#43142)

357fddf

Signed-off-by: Or Ozeri <oro@il.ibm.com>

[ROCm][CI] Stabilize 400 error return code for invalid schema inputs (v…

5940590

…llm-project#43016) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[ROCm] [DSv4] [Perf] Support DeepSeek v4 MTP (vllm-project#43385)

1806d1a

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>

Tuning script and configs for Triton Mamba SSU kernel (vllm-project#4…

d56285c

…3083) Signed-off-by: Banani Ghosh <bg2502@nyu.edu> Signed-off-by: Daniel Serebrenik <daserebrenik@nvidia.com> Co-authored-by: Banani Ghosh <bg2502@nyu.edu>

File system secondary tier implemented in python (vllm-project#41735)

d0a100c

Signed-off-by: Rotem Shavitt <rshavitt@gmail.com> Signed-off-by: Or Ozeri <oro@il.ibm.com> Co-authored-by: Or Ozeri <oro@il.ibm.com>

[Kernel] Add mhc_pre_big_fuse_with_norm_tilelang (vllm-project#43474)

b06813e

Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>

fix: MoE model using shared routed experts crashes on AMD GPUs (vllm-…

6cbe448

…project#42373) Signed-off-by: weizhou.lan@daocloud.io <weizhou.lan@daocloud.io>

[Docs] Reorganize offline inference docs. (vllm-project#43552)

1b26fa3

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

[Docker] Non-root support for vllm-openai; add opt-in vllm-openai-non…

3df1c7c

…root target (vllm-project#40275) Signed-off-by: TheDuyIT <nduy250299@gmail.com> Signed-off-by: dtnguyen <dtnguyen@nvidia.com> Co-authored-by: Claude <noreply@anthropic.com>

[Feat][KVConnector] Support DSV4 in SimpleCPUOffloadBackend (vllm-pro…

81252d4

…ject#42296) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai>

[Doc] Add section on escalating stalled contributions (vllm-project#4…

0c942c6

…3568) Signed-off-by: esmeetu <jasonailu87@gmail.com>

Reduce memory usage for granite_speech. (vllm-project#42933)

5c1aec3

Signed-off-by: Yihuki <wangbovbvb@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

[KV Connector] Handle Mooncake finish after preemption (vllm-project#…

873758c

…43281) Signed-off-by: Zhewen Li <zhewenli@inferact.ai> Co-authored-by: Zhewen Li <zhewenli@inferact.ai>

[Misc] Print accuracy value for PD tests even on success (vllm-projec…

716d529

…t#43583) Signed-off-by: NickLucche <nlucches@redhat.com>

[Kernel] Remove NormGateLinear (vllm-project#43554)

d400445

Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai>

[LoRA] Add one shot triton kernel For MoE LoRA (vllm-project#42290)

ec5de7f

Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>

[DeepSeek V4] Move MegaMoE input prep kernel to nvidia/ops (vllm-proj…

aa2b56f

…ect#43632) Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

[KV Connector][Bugfix] MooncakeStore: don't double-apply Eagle prune …

7966fc7

…in load_mask (vllm-project#43516) Signed-off-by: Dao Le <daole@inferact.ai> Signed-off-by: Dao Le <Dao007forever@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>

[KV Connector] Propagate MooncakeStore load failures (vllm-project#42788

c2a4005

) Signed-off-by: Dao Le <Dao007forever@gmail.com>

wang.yuqi (noooop) and others added 29 commits June 1, 2026 07:54

[Frontend] Resettle generative scoring entrypoint. (vllm-project#44153)

0910f7e

Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>

[Rust Frontend] Add InternLM2 tool parser (vllm-project#43481)

de21863

Signed-off-by: Will.hou <1205157517@qq.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

[Bugfix] fix wrong partial_rotary_factor calculation for bailing_moe …

8796838

…model. (vllm-project#43770) Signed-off-by: zzt <zengzetang.zzt@antgroup.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>

[XPU][CI] Fix test_audio_in_video flake by using module-scoped server…

bd0aecd

… fixture (vllm-project#44146) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com>

[Perf] Optimize cutlass fp8 scaled mm bypassing padding, 20% kernel p…

985c97a

…erformance improvement (vllm-project#43706) Signed-off-by: yewentao256 <zhyanwentao@126.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

[Feature] Add support for JetBrains' Mellum v2 code generation model (v…

023808c

…llm-project#43992) Signed-off-by: Madeesh Kannan <madeeswaran.kannan@jetbrains.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>

[Kernel][DSv4] Optimize sparse FP8 compressor kernels (vllm-project#4…

0357335

…4161) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>

[ROCm][CI] Fix and stabilize EAGLE3 acceptance tests (vllm-project#41294

fd9e91d

) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Micah Williamson <micah.williamson@amd.com> Co-authored-by: Micah Williamson <micah.williamson@amd.com>

[Rust Frontend] Support streaming generate endpoint (vllm-project#4…

182c67d

…3779) Signed-off-by: xunzhuo <xunzhuo@vllm-semantic-router.ai> Co-authored-by: Bugen Zhao <i@bugenzhao.com>

[Frontend][Core] Add sparse NCCL weight transfer support for in-place…

266b9d9

… updates (vllm-project#40096) Signed-off-by: Siddharth Bedekar <bedeksid@gmail.com> Co-authored-by: OpenAI Codex <codex@openai.com>

[BugFix][CI] Fix added _has_module tests (vllm-project#44248)

6f8b40a

Signed-off-by: Nick Hill <nickhill123@gmail.com>

[Test][BugFix] Fix double-BOS in PD+specdec acceptance test (vllm-pro…

e4cbc43

…ject#44234) Signed-off-by: Nick Hill <nickhill123@gmail.com>

[DSV4] Remove unncessary classes & functions (vllm-project#44246)

8c3cc98

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

[ROCm][CI] Skip unbacked dynamic shapes tests on PyTorch < 2.11 (vllm…

48c0d13

…-project#44256) Signed-off-by: JartX <sagformas@epdcenter.es>

[DSV4] Refactor RoPE initialization (vllm-project#44262)

517e74a

Signed-off-by: Woosuk Kwon <woosuk@inferact.ai>

[Bugfix][Mooncake] Release GPU pin on failed store in MooncakeStoreCo…

d68f0b2

…nnector (vllm-project#43742) Signed-off-by: Dao Le <Dao007forever@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>

[ROCm] Upgrade AITER to v0.1.13.post1 (vllm-project#44265)

2588ec4

Signed-off-by: Micah Williamson <micah.williamson@amd.com>

[Bugfix][CI] Normalize NIXL connector CUDA wheel installs (vllm-proje…

816cc73

…ct#44266) Signed-off-by: Alec Flowers <aflowers@nvidia.com>

[Refactor] Move unstreamed tool-arg flush from serving layer to parser (

9affc17

vllm-project#44017) Signed-off-by: sfeng33 <4florafeng@gmail.com>

[CI] Stabilize OpenAI schema fuzzing for malformed structural tags (v…

54d0c36

…llm-project#44131) Signed-off-by: Andreas Karatzas <akaratza@amd.com>

[BugFix][kv_offload]: Prevent offloading stale sliding window blocks (v…

480fada

…llm-project#42959) Signed-off-by: Or Ozeri <oro@il.ibm.com>

[XPU][Bugfix] Fix per_token_group_fp8_quant missing dummy args on XPU (…

a3a5a5e

…vllm-project#43930) Signed-off-by: Chaojun,Zhang <chaojun.zhang@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>

[MM][CG] Profile encoder CUDA graph pool memory (vllm-project#41714)

a045c74

Signed-off-by: JooHo Lee <jooho414@gmail.com>

[Bugfix] Convert Gemma4-MM ViT linear layers to vllm native impl (vll…

f91fb2f

…m-project#43798) Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> Co-authored-by: ZiTian Zhao <zitian.zhao@tencentmusic.com> Co-authored-by: B-201 <Joy25810@foxmail.com>

[Model Runner V2] Support zeroing freshly allocated KV blocks for hyb…

8a9eb40

…rid + fp8 KVCache (vllm-project#43990) Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>

[Model Runner V2] Use actual batch max_seq_len for attn metadata (vll…

1edfd09

…m-project#43991) Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>

[Refactor] Unify reasoning + tool-call parsing behind Parser.parse() (v…

68dafcc

…llm-project#44267) Signed-off-by: sfeng33 <4florafeng@gmail.com>

Merge upstream/main into macrodata main

abfeede

# Conflicts: # vllm/entrypoints/chat_utils.py Signed-off-by: Hynek Kydlicek <kydlicek.hynek@gmail.com>

Hynek Kydlíček (hynky1999) merged commit 9e72676 into main Jun 2, 2026
6 of 7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge upstream vLLM main#5

Merge upstream vLLM main#5
Hynek Kydlíček (hynky1999) merged 1314 commits into
mainfrom
codex/sync-upstream-main

Hynek Kydlíček (hynky1999) commented Jun 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

Hynek Kydlíček (hynky1999) commented Jun 2, 2026

Summary

Validation

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants