Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2084 commits
Select commit Hold shift + click to select a range
14f98cb
[Bugfix][XPU] store the pointer raw bit pattern instead of its numeri…
Yejing-Lai Sep 24, 2026
5963795
[Attention][CPU] Run Zen CPU encoder attention on zentorch SDPA (#54508)
Priyjain-amd Sep 24, 2026
bbd7c24
[MRV2] Validate MRV2 entrypoint logits processors (#57728)
taneem-ibrahim Sep 24, 2026
e430f74
[Bugfix][Rust Frontend] Prevent MM timing from enabling debug tracing…
reidliu41 Sep 24, 2026
cfd5c20
[XPU][UT] Align HF and vLLM inputs for Qwen2 embedding test by preven…
RyanMa29 Sep 24, 2026
f34a0e0
[CPU] Gate the AVX10.2 paths on compiler support (#58133)
ganeshr10 Sep 24, 2026
5747d45
[Perf][Frontend] Offload streaming derender detokenization (#57528)
KEYS-A15 Sep 24, 2026
f5a78f2
[Multimodal] Reuse the supplied tokenizer in the MiniMax-M3 VL proces…
liuzijing2014 Sep 24, 2026
dcfc17e
[XPU] enable XPU GRAPH by default (#51600)
zhenwei-intel Sep 24, 2026
721d0e5
[ROCm][DSv4.1][Perf] Emit MXFP8 from the sparse decode reduce and run…
Fangzhou-Ai Sep 24, 2026
cccf7e1
Remove `.gemini/` and `CLAUDE.md` (#58541)
hmellor Sep 24, 2026
484c211
[Bugfix][Pooling] Fix JinaVL label configuration and restore multimod…
LinzeShi Sep 24, 2026
d881812
[Chore] Use Transformers v5 names and drop redundant processor `use_f…
hmellor Sep 24, 2026
3c42385
[Refactor] Remove dead or duplicate tests (#58446)
yewentao256 Sep 24, 2026
26f49e3
[Perf][Attention] Bound FlashInfer prefill dequantization scratch (#5…
eopXD Sep 24, 2026
a7d203e
fix(config): apply presence_penalty/frequency_penalty from override-g…
hclsys Sep 24, 2026
344fcc2
[Bugfix] Resolve the Hub revision once per repo (#56092)
Wauplin Sep 24, 2026
d636456
[Frontend] Remove the slow tokenizer mode (#58545)
hmellor Sep 24, 2026
09fe178
Revert "[DSpark] Support pipeline-parallel targets in aggregated serv…
njhill Sep 24, 2026
8b84e15
[transformer] RMSNorm matching for alternative rsqrt (#54461)
bohnstingl Sep 24, 2026
5b3f280
[Bugfix] Count unsplit Idefics3 image patches (#48760)
nightcityblade Sep 24, 2026
074d57f
[Bugfix] Keep JIT warmup under enforce-eager when fault tolerance is …
njhill Sep 24, 2026
e30559b
[Core] Skip JIT monitor when JIT warmup is disabled (#58590)
njhill Sep 24, 2026
e773247
[Fast Start] Wait for weight cache daemon readiness (#58370)
UNIDY2002 Sep 24, 2026
cd06e81
[Bugfix][Quantization] Add Humming to the W4A8 (INT4xFP8) MoE oracle …
mgoin Sep 24, 2026
58aa2f1
[Refactor] Move auxiliary files out of the repository root (#58572)
mgoin Sep 24, 2026
90a9515
[Cleanup] Remove online quantization support in `fp8.py` in favor of …
fxmarty-amd Sep 24, 2026
92e9c6b
[ROCm] Fix misrouting race-condition in multi-decode P/D disagg with …
vcave Sep 24, 2026
ab3de6e
[Perf] DiffusionGemma: constrained reads over the request's logprob_t…
mmastrac Sep 24, 2026
0908116
[Bugfix] Pass quant_config to DiffusionGemma's ParallelLMHead (#48521)
AARONKANG04 Sep 24, 2026
e33de82
[ROCm][CI] skip the ROCm MRV1 default where MRV1 cannot serve the con…
stefankoncarevic Sep 24, 2026
04730e8
[DFlash] Capture the context K/V precompute in the draft CUDA graph (…
Juntian777 Sep 24, 2026
36f94d5
[Bugfix][Outlines] Fix EOS termination and unconstrained masks after …
njhill Sep 24, 2026
e6c07ea
[Bugfix][KV Cache] Fix incremental multimodal block hashing (#51694)
CZT0 Sep 24, 2026
7edb27f
[XPU][CI] enable prompt embeds tests on XPU (#58283)
faaany Sep 24, 2026
f5b40fc
[CI] Report to CRCR after all jobs finish, gated on the build's long …
atalman Sep 24, 2026
6a86bf6
[PD][PushConnector] Record last activity of remotes on the D side (#5…
snadampal Sep 24, 2026
bd4cb3f
[BUGFIX] fix ovis2_5 multimodal tokens (#52623)
microslaw Sep 24, 2026
f6aa291
[Bugfix][Core] Keep every multimodal feature in the partial-block KV …
haosenwang1018 Sep 24, 2026
9ac3b7c
[ROCm][CI] Mirror the DSv4-Flash disaggregated DP EP group on MI355 (…
stefankoncarevic Sep 24, 2026
4f72bd0
[Bugfix][Quantization] Give LM heads standard linear metadata (#58444)
mgoin Sep 24, 2026
d5051ab
[Bugfix][Mamba] Restore prompt-tail prefix-cache hits with MTP (#58368)
netanel-haber Sep 24, 2026
71891bd
[Perf] Parallelize registered CUDA Triton kernel warmup at startup (#…
mgoin Sep 24, 2026
7a94066
[KV Connector] Fix DecodeBench fp8 fill values and add a startup fill…
liuzijing2014 Sep 24, 2026
7833ff2
Release prompt_embeds tensor when its InputBatch slot is freed (#57988)
khushali9 Sep 25, 2026
b3a87b2
[Bugfix][KVConnector] Finalize saves on steps without a forward (#57775)
ivanium Sep 25, 2026
064bc74
[Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 an…
sammaji Sep 25, 2026
3972475
[Bugfix] GLM-5.3-Flash: launch the kpool paged MQA logits in the varl…
ivanium Sep 25, 2026
a811738
[Bugfix] Accept EOS after grammar finish in outlines backend; reject …
SIDDARTHAREDDY8 Sep 25, 2026
34ee85d
[Perf] Batch Mamba2 prefill SSM state saves, removing GPU<->CPU syncs…
samuelkim7 Sep 25, 2026
155b988
[CI] Run DFlash2 NVFP4 acceptance test on B200; skip it on H200 35GB …
khluu Sep 25, 2026
7dbd0a8
[Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybr…
njhill Sep 25, 2026
ade1056
[Kimi-K3][Perf] Dispatch GEMM for vision patch embedder (#58527)
gau-nernst Sep 25, 2026
16070ed
[Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset f…
gau-nernst Sep 25, 2026
3652f35
[Bugfix][Quantization] Refresh online NVFP4 scales before reload post…
S1ro1 Sep 25, 2026
609731e
[Perf][DSv4.1] Restore the fused query RMSNorm + MXFP8 quantization p…
Juntian777 Sep 25, 2026
32fc823
[Bugfix][LogitsProcessor] Validate ':' separator in custom logits pro…
100milliongold Sep 25, 2026
65c5e24
[ROCm][CI][AITER Coverage] Harden MoE sorting-backend/dispatch env-va…
divakar-amd Sep 25, 2026
34e104f
[gRPC] Fix ping tolerance so long non-streaming RPCs are not dropped …
gongwei-130 Sep 25, 2026
055f150
[Perf][Rust Frontend] Make histogram observations lock-free (#58574)
BugenZhao Sep 25, 2026
3059155
[ROCm][Perf] MXFP8 GEMM on native 32x32 block scales for gfx950 (#58510)
Fangzhou-Ai Sep 25, 2026
48d8880
[Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph…
Juntian777 Sep 25, 2026
6936e77
[Perf][Engram] Serialize offloaded lookups and pack host tables into …
Juntian777 Sep 25, 2026
f55419c
[Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxf…
samuelkim7 Sep 25, 2026
4d79d5c
[ROCm] Cut 69 wasted contiguous copies per decode step from the skinn…
fululi12 Sep 25, 2026
8b365ff
[ROCm][Build] Filter crate tags from vLLM version detection (#57744)
reidliu41 Sep 25, 2026
442bd00
[Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM10…
zyongye Sep 25, 2026
b05f7d5
[PP][XPU]Add the flag to control microbatch feature on MRV2+PP (#55145)
yisustc Sep 25, 2026
44257c2
[Feature] Triton kernel dispatcher (#43048)
wangxiyuan Sep 25, 2026
4013f3a
[ROCm] Fix CI runtime and tests for MI355 DPX (#58244)
mkunredd Sep 25, 2026
fa94450
[CI][ROCm] Prevent Model Executor apt stalls (#58607)
AndreasKaratzas Sep 25, 2026
267eee5
[Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497)
mrodden Sep 25, 2026
8a26991
[MM] Add Triton kernel for mm_input_normal. (#56798)
noooop Sep 25, 2026
1f0bc49
[Structured Outputs] Parse Lark grammars natively in the xgrammar bac…
BugenZhao Sep 25, 2026
35f92d2
[ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-…
valarLip Sep 25, 2026
5a2ed2e
[Frontend] Handle Disable Thinking in /v1/messages (#58613)
robertgshaw2-redhat Sep 25, 2026
3a58aeb
[Bugfix] Stop leaking the internal field name in the max_tokens valid…
shallow10 Sep 25, 2026
29468dd
[Bugfix][KV Cache][MLA] Align packed block strides for V3.2 sparse ML…
200lz Sep 25, 2026
5ff3bbf
[Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA…
zyongye Sep 25, 2026
24928a4
[UX][Frontend] Introduce `vllm preload` cli for fast restart (#56680)
Isotr0py Sep 25, 2026
378504a
[MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635)
zyongye Sep 25, 2026
afea5c2
[Docs] Add return annotation to `fused_mm_input_norm_triton` (#58687)
hmellor Sep 25, 2026
fa6c407
[CI] Shard (H100) Helion Kernels five ways (#58645)
Thangnguyenvn98 Sep 25, 2026
45d56c0
[Core] Model console logging as CLI configuration (#57205)
markmc Sep 25, 2026
25b0add
[ROCm][CI] Pass weight_shape in MXFP8 block32 linear tests (#58698)
djramic Sep 25, 2026
73a78e6
[SpecDecode] Add LiLiCorr drafter (#57934)
askliar Sep 25, 2026
234df71
[MRV2] Minor model_runner.py code cleanup (#58610)
njhill Sep 25, 2026
50f7808
[Bugfix] Fix generative scoring body cancellation (#57729)
taneem-ibrahim Sep 25, 2026
bb35e25
[Pooling] Preserve reranker tokenization with document limits (#57666)
taneem-ibrahim Sep 25, 2026
f550ad8
[Kernel][Perf] Register-resident path for per-token-group 8-bit quant…
chuan932 Sep 25, 2026
38cc054
[Perf][Kernel] Vectorized flat abs-max for dynamic per-tensor FP8 qua…
Monishver11 Sep 25, 2026
a44d7b5
[Core][Logging] Fix JSON logging process decoration (#57957)
markmc Sep 25, 2026
6491f48
[Bugfix] Stop allocator fragmentation from shrinking the KV cache dur…
robertgshaw2-redhat Sep 25, 2026
f339e0e
[Bugfix][Frontend] Keep logprobs of parser-suppressed streaming chunk…
errmakov Sep 25, 2026
2edbb32
[Bugfix][GLM-5.3-Flash] SM90 sparse MLA: index_kpool mismatch leads t…
mmastrac Sep 25, 2026
f83ea61
[Benchmark] Record model_id in bench latency/throughput --output-json…
YashasviChaurasia Sep 25, 2026
2617fe9
[Bugfix][GLM-5.3-Flash] kpool corruption with speculative decoding (#…
mmastrac Sep 25, 2026
c88f482
[GLM5.3 Perf] Optimize glm 5.3 metadata op, 1.6~4.8x kernel level per…
yewentao256 Sep 25, 2026
c89b5b4
[Bugfix][KV Connector] Reap expired NIXL leases behind a heartbeated …
gokay-ai Sep 25, 2026
1417022
[ROCm][Kimi-K3] Optimize low-concurrency speculative KDA (#58045)
jiacao-amd Sep 25, 2026
b217575
[ROCm][CI] Cover the AITER MQA logits dispatch on gfx950 (#58724)
stefankoncarevic Sep 25, 2026
1b922ff
[ROCm][CI] Add quantized MoE serving test for gfx950 (#58748)
stefankoncarevic Sep 25, 2026
e3dd5b5
[ROCm][CI] Test AMD DeepSeek V4 MoE routing against a PyTorch referen…
stefankoncarevic Sep 25, 2026
353e491
[Perf] DiffusionGemma: one-pass sampler statistics kernel (#58226)
mmastrac Sep 25, 2026
22bbe3f
[ROCm][CI] Expand single-GPU coverage on MI355 DPX (#57599)
sheralskumar Sep 25, 2026
2bc902e
[CI] [MRV2] Restore MRV2 pp dp coverage (#57735)
taneem-ibrahim Sep 25, 2026
2d7dd0e
[Bugfix][CI] Report subprocess test skips as skips, not passes (#58701)
stefankoncarevic Sep 25, 2026
31cc226
[ROCm][CI] Run the MLA attention+quant fusion test on ROCm (#58717)
stefankoncarevic Sep 25, 2026
e440b75
[ROCm] Bump torch 2.13, triton 3.8, torchaudio, torchvision (#50605)
Rohan138 Sep 25, 2026
83db8c8
[Feature][Frontend] Add granite_thinking_parser reasoning parser for …
yousafshah Sep 25, 2026
7871963
[Bugfix][Frontend] Respect max_output_tokens in the Harmony tool-call…
errmakov Sep 25, 2026
de2d478
[Bugfix][Frontend] Document 404 response for `/generative_scoring` (#…
njhill Sep 25, 2026
8b47e8b
[Perf][Frontend] Defer reasoning usage recounts for non-continuous ch…
positive666 Sep 25, 2026
b6761e8
[Bugfix] Default missing detail for Responses API input images (#57241)
yzong-rh Sep 25, 2026
4ccfe12
[watermarking] golden tests for backwards compatibility (#56809)
TQCB Sep 25, 2026
e55d076
[Bugfix] Don't drop the rest of the allocator config when toggling ex…
okorzh-amd Sep 25, 2026
614b55f
[AuxOutput] Only require Model Runner V2 on GPU platform (#58205)
lk-chen Sep 25, 2026
6ec2f84
[Pooling] Preserve BERT-family heads for raw logits (#57664)
taneem-ibrahim Sep 25, 2026
b040321
fix: perf: use startswith(x, i) instead of string slicing to avoid O(…
Ricardo-M-L Sep 25, 2026
974cb65
[Bugfix] V1: fix allowed_token_ids_mask aliasing in InputBatch.swap_s…
huthvincent Sep 25, 2026
9bf44c4
[Bugfix][Reasoning] Count Kimi K3 reasoning tokens (#58372)
elvircrn Sep 25, 2026
da1aaec
[CI] Split (H200 MIG/MI300) Basic Correctness into named jobs (#57054)
Thangnguyenvn98 Sep 26, 2026
fdcce47
[Bugfix][Frontend] Fix Inkling tool name leaking into content after r…
baljinderhothi-cohere Sep 26, 2026
8a23646
[Core] Bound draft-token RPC waits by the execute-model timeout (#58779)
vschandramourya Sep 26, 2026
4bb804c
[Perf][DSv4.1] Shard the Engram wkv projection across TP ranks (#58678)
ShuoleiWang Sep 26, 2026
b6d2b4f
[RL] Add sharding-aware NCCL M2N weight transfer (#51520)
kwen2501 Sep 26, 2026
5840d95
[Bugfix][ROCm] AMD-Quark mixed-precision DeepSeek-V4.1 support (#57071)
xiao-llm Sep 26, 2026
ddd6fbc
[Bugfix][Frontend][Rust Frontend] Update DeepSeek V4.1 Flash reasonin…
zhecfy Sep 26, 2026
3b4566c
[CI] Only isolate the registry tests that need a fresh process (#58764)
aarushjain29 Sep 26, 2026
a4eb3f2
[Spec Decode] Enable Gemma4 DSpark adaptive verification with FlashIn…
zixi-qi Sep 26, 2026
2f41e00
[CI] Split (B200) Miscellaneous Kernels into mHC, FLA Ops and Misc na…
Thangnguyenvn98 Sep 26, 2026
31f2e70
[Security] Gate per-request multimodal processor kwargs (#58830)
jperezdealgaba Sep 26, 2026
2bb7605
[Elastic EP] Fix EPLB load statistics during scaling (#58473)
itayalroy Sep 26, 2026
bf93e1f
[Refactor] Remove dead tests utils (#58803)
yewentao256 Sep 26, 2026
ad6817b
[Perf][MoE] Index expert mapping lookups in RoutedExperts.load_weight…
Willian-Zhang Sep 26, 2026
d64f277
[Bugfix] Fix Anthropic Thinking Disabled with P/D (#58786)
robertgshaw2-redhat Sep 26, 2026
dfab504
[Bugfix][Frontend] Detect Anthropic inline-system merge against the r…
robertgshaw2-redhat Sep 26, 2026
5113cf9
[Mypy] Fix mypy typing for Qwen and Qianfan models (#58046)
ashraf-bhuiyan Sep 26, 2026
79ac28c
[CI] Reduce CUDA graph mode test overhead (#58749)
mgoin Sep 26, 2026
3576691
[GLM5.3 Bug] Fix sparse indexer attn topk backend selection (#58594)
yewentao256 Sep 26, 2026
0239bd2
[CI] Stabilize batch submission in full CUDA graph tests (#58810)
AndreasKaratzas Sep 26, 2026
7d8c5fe
[Bugfix][DSV4.1] Avoid host sync in ViT CUDA graph replay metadata (#…
khluu Sep 26, 2026
379e9a1
[Security] Harden message sanitization (#58832)
DarkLight1337 Sep 26, 2026
7787112
[Kernel][DSV4.1] Fuse MoE finalize into the TP all-reduce + mHC bound…
zyongye Sep 26, 2026
927c87b
[PCP][DCP] Support DCP target model with non-DCP Dspark (#56723)
LucasWilkinson Sep 26, 2026
4be061c
[Kernel] Bump FlashKDA to keep the recurrent state in fp32 (#58846)
simon-veitner-redhat Sep 26, 2026
cd1daf5
[Metrics][KV Offload] Add Prometheus metrics for SimpleCPUOffloadConn…
mevince Sep 27, 2026
eb0f2ca
[mooncake] support CUSTOM_MEM_POOL in vllm (#49300)
voidxb Sep 27, 2026
7a877ae
[Qwen3.8-Flash-Next] Avoid memory fragmentation in QSA indexer logits…
gau-nernst Sep 27, 2026
0c87a19
[Bugfix] Disable sequence parallelism / async TP under batch invarian…
LioEinaudi Sep 27, 2026
3137ff0
[ROCm][Kimi-K3] Make VLLM_ROCM_USE_AITER_MOE_SITUV2 select a4w4/a8w4/…
hongxiayang Sep 27, 2026
fff0326
[Bugfix][NIXL] Release a dead peer's NIXL state without waiting for T…
YannikHinteregger Sep 27, 2026
fba4739
[Bugfix] V1: clear stale allowed_token_ids mask in InputBatch.condens…
V-3604 Sep 27, 2026
a9eafde
[Perf][DSv4.1] Fuse small-batch WO-A with inverse RoPE and MXFP8 quan…
Juntian777 Sep 27, 2026
24c9772
[Attention][CPU] Use zentorch SDPA for CPU MLA prefill (#54967)
Rakul-Chauhan Sep 27, 2026
c8d7a7d
[Perf][Attention] Remove D2H sync from FlashInfer SM90 sparse MLA pla…
NickLucche Sep 27, 2026
924707f
[CI] Split (H200 MIG 35GB) Spec Decode Speculators + MTP into 4 named…
Thangnguyenvn98 Sep 27, 2026
44af287
[ROCm][Perf] Enable layer-aware CSA2 multi-stream overlap for DeepSee…
shen-shanshan Sep 27, 2026
0376f81
[Perf][Pooling] Avoid blocking seq_lens GPU-to-CPU copy for pooling i…
frankwang28 Sep 27, 2026
2b9b55c
[Bugfix] Support repsonse_format + tool_choice=auto (#56086)
arpera Sep 27, 2026
e790015
[Fast Start] Add `/health` endpoint for the weight cache daemon (#58552)
UNIDY2002 Sep 27, 2026
231fdb8
[Perf][MoE] Use fused MiniMax2 routing with non-unit routed scaling (…
ZJY0516 Sep 27, 2026
187c81b
[ROCm][Bugfix] Fall back to default GEMM for CPU tensors on ROCm buil…
Fangzhou-Ai Sep 27, 2026
386ac25
[Bugfix][KV Connector] Retry Mooncake bootstrap registration on timeo…
LCAIZJ Sep 27, 2026
73859fe
[Frontend] Switch Python Harmony dependency to oss-harmony (#55128)
PeganovAnton Sep 27, 2026
31afa4f
[ROCm] Bump AITER to v0.1.23 (#58867)
Fangzhou-Ai Sep 28, 2026
3184226
[Bugfix][Frontend] Count Responses reasoning tokens per tool round (#…
sfeng33 Sep 28, 2026
39d4953
[KimiViT][Perf] Fuse per-layer QK RoPE into one in-place kernel (#58651)
gau-nernst Sep 28, 2026
b9e2903
[Mypy] Fix mypy typing for Ultravox and Unlimited-OCR models (#58239)
ashraf-bhuiyan Sep 28, 2026
4d07e90
[Bugfix][Frontend] Sample batched chat completions from the adjusted …
sfeng33 Sep 28, 2026
027b6f3
[Bugfix][Frontend] Use a fresh parser per choice in non-streaming cha…
sfeng33 Sep 28, 2026
53d2e16
[Bugfix][EPD] Skip sampling for encoder-only async steps (#58490)
gty111 Sep 28, 2026
5eaa50f
[CPU] Build CPU wheels on Ubuntu 22.04 with AMX-FP8 support (#58515)
zhejiangxiaomai Sep 28, 2026
77fbd9e
[Test][ROCm] Stabilize the mixed OLMoE LoRA test (#58945)
AndreasKaratzas Sep 28, 2026
4a190dc
[Model] Enable LoRA support for RobertaForSequenceClassification (#58…
jz-yolo Sep 28, 2026
6f65fde
[Bugfix][Frontend] Apply Harmony adjust_request in batched chat compl…
sfeng33 Sep 28, 2026
55de40a
[Minimax-M3] Add Encoder CUDA graph support (#58673)
gau-nernst Sep 28, 2026
dd3bc9c
[Perf][MRV2] Reuse Mamba/GDN metadata across KV cache groups (#58762)
lucamotz Sep 28, 2026
0da1266
[Bugfix] Fix the two multimodal root tests that fail on main (OpenPan…
khluu Sep 28, 2026
af7f948
[Benchmark] Add Responses API backend to vllm bench serve (#54628)
QHarshil Sep 28, 2026
2407f40
[CI] Allowlist-shrink batch 1: wire 17 root-level tests + drop 5 stal…
wjabbour Sep 28, 2026
1668e59
[CPU] Use accelerator memory API in DiffusionGemma (#58964)
zhejiangxiaomai Sep 28, 2026
fd2f3fc
[CPU] Add video inferencing via torchcodec on s390x (#58693)
R3hankhan123 Sep 28, 2026
4694145
[Skills] Update kernel-microbenchmark to include ROCm (#58646)
gau-nernst Sep 28, 2026
004e37e
[Misc] Name each backend and its kernel block sizes in block-size err…
jayzuccarelli Sep 28, 2026
413ae4b
[CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integration (Pooli…
Thangnguyenvn98 Sep 28, 2026
66d2d4b
[CI] Split Dynamic Shapes out of (H200 MIG 35GB) PyTorch Compilation …
Thangnguyenvn98 Sep 28, 2026
a2be4d3
[Bugfix][Qwen4Exp] Release the profiling KV cache held by QSA key vie…
lucifer1004 Sep 28, 2026
ccfd1ce
[CPU] Include vLLM Recipes tooling in release image to deploy models …
louie-tsai Sep 28, 2026
2522793
Revert "[CI] Shard (H200 MIG 35GB / MI355 DPX) Entrypoints Integratio…
khluu Sep 28, 2026
d6cce94
[Perf][Qwen4Exp] Fuse HC down projection and SiLU on NVIDIA (#58957)
gau-nernst Sep 28, 2026
8cc9aa5
[Bugfix][Model] Gemma4: register aliased embedding scalars as buffers…
yannicks1 Sep 28, 2026
f6877f5
[Bugfix][Spec Decode] Implement get_top_tokens() on the ROCm DeepSeek…
BaoYunkai Sep 28, 2026
20b52e9
[Perf][Qwen3.8] Reduce PLE metadata construction overhead (#58114)
ZJY0516 Sep 28, 2026
7ba3df6
[ROCm][Refactor] Move DeepSeek-V4/V4.1 multi-stream overlap gate to R…
shen-shanshan Sep 28, 2026
522101f
[Multimodal] Avoid extra d2d for encoder cudagraph with fused input n…
Isotr0py Sep 28, 2026
d7f5722
[Bugfix][Logging] Preserve application log record factories (#58747)
mgoin Sep 28, 2026
cd63bf9
[Bugfix] Use the correct repository revision for secondary artifact l…
KernelClint Sep 28, 2026
a7ff443
[ROCm][Perf] Kimi-K3 Enable sharded latent MoE up-projection under EP…
xaguilar-amd Sep 28, 2026
b721a4c
[Feature] Add fixed-token prefill scoring (#54335)
aoshen02 Sep 28, 2026
953f90d
[LoRA] Support variable num_labels for sequence classification (#57766)
linitra24 Sep 28, 2026
6b24bd8
[ROCm][Perf] Replace torch.topk in DSA candidate block selection (#58…
Fangzhou-Ai Sep 28, 2026
b0d2b85
[ROCm]Keep LMCache OpenTelemetry on the image's 1.40 stack (#59056)
micah-wil Sep 28, 2026
94d1462
[Bugfix][Kimi-K3] Refresh DSpark context KV cache pointers after the …
okorzh-amd Sep 28, 2026
6d3ea3c
[KVConnector][NIXL] Support packed MLA KV layouts in pipeline-paralle…
zixi-qi Sep 28, 2026
c01368f
[Quant] Use canonical N-first weight format for CT WNA16 MoE (#52798)
HDCharles Sep 28, 2026
6c300dd
[CI][Bugfix] Relax packed_qk_rope_ correctness test to one ULP (#59008)
stefankoncarevic Sep 28, 2026
7a160c6
[ROCm][CI] Pin OpenTelemetry to LMCache's cap in the ROCm images (#59…
Rohan138 Sep 28, 2026
5ba2e30
[ROCm][CI] Increase timeout for Entrypoints Unit (#59076)
djramic Sep 28, 2026
25624f6
[ROCm][MLA] Add an AITER ASM round-robin decode route for DCP multi-t…
xiaohuguo2023 Sep 28, 2026
7644135
[Bugfix] Don't sync-police or retry FlashInfer all-reduce workspace c…
khluu Sep 28, 2026
2f4a975
[Bugfix] Fix resumable request + async scheduling handoff race (#58259)
yzong-rh Sep 28, 2026
ef42093
[Model Runner V2][Spec Decode] Support spec decode with draft model (…
wxsIcey Sep 28, 2026
e7c9036
[Perf][MRV2] Allow FULL decode graphs for one-token prompt tails (#58…
njhill Sep 28, 2026
2840ca7
[Bugfix][Scheduler] Preserve logprobs across streaming continuations …
i-m-aditya Sep 28, 2026
2619b6b
[Mypy] Fix mypy typing for Voxtral and vision models (#58251)
ashraf-bhuiyan Sep 28, 2026
42dbe45
[Core] Rework scheduler `skipped_waiting` queue (#58947)
njhill Sep 28, 2026
fedbc3b
[Bugfix][MRV2][Spec Decode] Reject draft slots that were never propos…
zixi-qi Sep 28, 2026
beef814
[ROCm][CI] Add missing test coverage for upstream parity (#50519)
AndreasKaratzas Sep 28, 2026
d28795f
[Bugfix][Scheduler] Refresh max tokens for streaming continuations (#…
i-m-aditya Sep 28, 2026
3d5f4d4
[Perf][Spec Decode] Avoid triton recompiles in the acceptance estimat…
TheEpicDolphin Sep 28, 2026
ee09ef4
[CI/Build] Skip the snapshot runtime on CUDA 12.x images (#59118)
khluu Sep 29, 2026
3f1c301
[XPU][CI]Skip test_abort_timeout_on_prefiller in nightly (#58307)
zxd1997066 Sep 30, 2026
69e838e
[Bugfix][Engram] Keep THP tables private when resolving shared memory…
yuzhouo7 Sep 29, 2026
0be3508
[ROCm][CI] Expand MI355 mirrors and route MIG-sized jobs to DPX (#59137)
AndreasKaratzas Sep 29, 2026
7a7283a
[Bugfix][Mamba] Keep the prompt-end prefill checkpoint under sparse r…
JaredforReal Sep 29, 2026
7f99308
[Core][BugFix] Tag prefix-cache extra keys by source (#51899)
KernelClint Sep 30, 2026
1d17b13
[Bugfix][Frontend] Reject LoRA adapters named after a served model (#…
njhill Sep 30, 2026
bffd162
[Dependency] Upgrade FlashInfer to 0.7.0.post1 (#59323)
wzhao18 Sep 30, 2026
25a029c
[Bugfix][HiSparse] Resolve MTP verification rows with a union residen…
MatthewBonanni Sep 30, 2026
197712e
[Bugfix][HiSparse] Never allocate GPU pages without host backing (#59…
LucasWilkinson Sep 30, 2026
3700ae3
[CPU][Whisper] Support W4A16 quantized Whisper on the CPU WNA16 kerne…
harshaladhav-amd Sep 30, 2026
cc28ed1
[Core] Bound UniProc EngineCore startup threads to available CPUs (#5…
AndreasKaratzas Sep 30, 2026
a5738bf
[KV-Offloading][TP] : Expand replicated_layout detection to multi-gro…
varun-sundar-rabindranath Sep 30, 2026
65182ef
[Bugfix][HiSparse] Preserve host prefix publication after request com…
eopXD Sep 30, 2026
f9f8c13
[Bugfix][HiSparse] Adopt GPU prefix copies after the hit's allocation…
MatthewBonanni Sep 30, 2026
1ea61d3
[Bugfix][HiSparse] Stop the host pool feeding device KV cache residen…
eopXD Sep 30, 2026
2e3e990
[Bugfix][Core] Fix mamba prefill checkpoint block reservation and pro…
kliuae Sep 30, 2026
41b8365
[Core] Include the LoRA path in prefix-cache block hashes (#59335)
njhill Oct 1, 2026
024c957
[Perf][PP] Skip sampled-token broadcasts whose requests leave the eng…
LostFox11 Oct 1, 2026
443ad8e
[CPU][Zen] Add DA8W4 (W4A8) int4 support for dense and MoE layers (#5…
ganeshr10 Oct 1, 2026
f426292
[Model Runner V2] Support randomized dummy inputs (#58411)
robertgshaw2-redhat Oct 1, 2026
a9ccd6d
[Bugfix][HiSparse] Fix a chunked-prefill preemption livelock (#59494)
MatthewBonanni Oct 1, 2026
77cca24
[Bugfix][HiSparse] Size the KV cache from the groups HiSparse allocat…
MatthewBonanni Oct 2, 2026
31ebe18
[Bugfix][HiSparse] Fix MTP acceptance collapse under FULL graphs with…
LucasWilkinson Oct 2, 2026
f647635
[CI] Drop a test that depends on #57930 from the #59309 backport
khluu Oct 2, 2026
967be6b
[Bugfix] Fix minimax-m3 multimodal processor compatability with Trans…
Isotr0py Oct 2, 2026
db9527a
[Misc] Add Transformers version upper bound in requirements (#59614)
Isotr0py Oct 2, 2026
e543687
Forward-port the tkv/turbo-attn vLLM seam onto upstream v0.31.0
arbi-dev Oct 7, 2026
0215372
Merge origin/main (#30) into upgrade/v0.31.0-pool (tree unchanged)
arbi-dev Oct 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion PROTOCOL.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,40 @@ per N layers in each block (`get_tensor_slots`) and gives every fused layer a
view of that same page. Without it the summed page is charged once per layer
and usable capacity drops N-fold.

### Separate state pool for hybrid models — `vllm/v1/core/`, `vllm/v1/worker/`

A hybrid model's O(1) recurrent layers (Mamba/GDN) hold a fixed number of
blocks per running request, never one per token. Upstream puts them in the
attention pool: every recurrent layer gets a tensor of `num_blocks` state
pages and its page is unified with the attention page, which spends most of
the KV budget on state no request can use (Qwen3.5-0.8B at 10 GB: 5,728
tokens shared vs 355,696 split; turbo-attn ATTRIBUTION §7.3).

The seam gives those groups their own pool when it is safe to
(`_mamba_pool_eligible`: Mamba cache mode `none`/`align`, no HiSparse, no KV
connector, no context parallelism, no hidden-state layers):

- **Planning** (`kv_cache_utils.py`): the recurrent pages are not unified
with attention (attention layers are unified among themselves); the state
pool is sized `max_num_seqs * sum(per-request blocks per group) + 1` and,
with a sampler-warmup reserve (`VLLM_SAMPLER_RESERVE_MIB`), comes off the
budget before attention is planned, so auto-fit, the override and the
admission check all see only the attention pool. The two pools are two
regions of the one backing allocation, each laid out exactly as upstream
lays out a shared pool (`_layout_kv_cache_tensors`), so groups within a
pool still overlay each other and the pools never alias.
`KVCacheConfig.mamba_pool_num_blocks` / `mamba_pool_offset`,
`KVCacheTensor.num_blocks`.
- **Scheduler** (`kv_cache_coordinator.py`, `kv_cache_manager.py`): a second
`BlockPool` for the recurrent groups; admission checks each pool against its
own free blocks (watermark and reservations count main-pool blocks);
events and usage cover both. Frees route by `KVCacheBlock.pool`, which
upstream already does.
- **Worker** (`worker/utils.py`, `worker/gpu/*`): the two pools' block ids
overlap, so CoW copies carry state-pool ids shifted past `num_blocks` and
apply only to the caches in that pool's region; profiling dummy contexts
and warmup number each pool's blocks separately.

---

## Sync workflow when moving to a new upstream release
Expand All @@ -113,7 +147,7 @@ and usable capacity drops N-fold.
extension point that covers a hook, move the capability into the plugin
and drop the hook.
3. `pytest tests/v1/core/test_kv_seam_invariants.py
tests/v1/core/test_aggregated_layer_count.py
tests/v1/core/test_aggregated_layer_count.py tests/v1/core/test_mamba_pool.py
tests/v1/attention/test_mla_wrapper_selection.py
tests/v1/worker/test_gpu_worker.py -k hook
tests/engine/test_arg_utils.py -k plugin_kv_cache_dtype` (CPU).
Expand Down
8 changes: 2 additions & 6 deletions tests/engine/test_arg_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -1081,9 +1081,7 @@ def test_plugin_kv_cache_dtype_auto_selects_turbo_attn_backend():
)
engine_args = EngineArgs.from_cli_args(args)
vllm_config = engine_args.create_engine_config()
assert vllm_config.attention_config.backend == (
AttentionBackendEnum.TURBO_ATTN
)
assert vllm_config.attention_config.backend == (AttentionBackendEnum.TURBO_ATTN)

# 2. Plugin dtype + explicit --attention-backend → user choice honoured.
args = parser.parse_args(
Expand All @@ -1098,9 +1096,7 @@ def test_plugin_kv_cache_dtype_auto_selects_turbo_attn_backend():
)
engine_args = EngineArgs.from_cli_args(args)
vllm_config = engine_args.create_engine_config()
assert vllm_config.attention_config.backend == (
AttentionBackendEnum.FLASH_ATTN
)
assert vllm_config.attention_config.backend == (AttentionBackendEnum.FLASH_ATTN)

# 3. Builtin dtype "auto" → no auto-override (stays None).
args = parser.parse_args(
Expand Down
Loading