generate invoking doesn't require detokenization for beam search - #33563

Closed

gameofdimension wants to merge 102 commits into

vllm-project:mainfrom

gameofdimension:patch-1

Commits on Feb 5, 2026

detokenize is unnecessary

gameofdimension
authored and
felix01.yu
committed
b98ce64
[Model] Use explicit types in `get_generation_prompt` (#33551)

DarkLight1337
authored and
felix01.yu
committed
4b2d503
offline beam search doesn't require detokenization either

gameofdimension
authored and
felix01.yu
committed
dfd512f
fix[ROCm]: Remove unconditional aiter import (#32902)

rabi
authored and
felix01.yu
committed
9878396
[MoE] Enable Shared/Routed Overlap For Latent MoE (Nemotron-H) (#32790)

danielafrimi
authored and
felix01.yu
committed
5b6bc9b
[Bugfix] Enable Kimi k25 processor test (#33562)

Isotr0py
authored and
felix01.yu
committed
f5dc8c0
move spec decode slow test to test_areas.yaml (#33365)

shanjiaz
authored and
felix01.yu
committed
525b318
[Feature][Core] Support Fabric detection to adapt the MNNVL protocol for the GB series (#33540)

authored and
felix01.yu
committed
f56f36b
[Refactor] Move profiling methods to MM budget (#33559)

DarkLight1337
authored and
felix01.yu
committed
8c818aa
[CI] Add DeepSeek V3.2 nightly eval (#33566)

MatthewBonanni
authored and
felix01.yu
committed
5941e46
[Model] Use mm_position to compute mrope positions for GLM-4.xV (#33039)

KKSK-DON
authored and
felix01.yu
committed
a89e73a
Remove incorrect tokenizer info test (#33565)

hmellor
authored and
felix01.yu
committed
35a1923
Update huggingface-hub again (#33567)

hmellor
authored and
felix01.yu
committed
2588853
Reduce the kernel overhead when num of active loras is smaller than max loras. Multiple cuda graphs are captured for each num of active-loras. (#32005)

yugong333
authored and
felix01.yu
committed
a37cdde
[UX] Format attention backend log line (#33570)

MatthewBonanni
authored and
felix01.yu
committed
49a44bb
fix memory for online fp8 quantization with streaming weight load (#31914)

vkuzo
authored and
felix01.yu
committed
1b9e3ff
fix cutlass_3x_gemm_fp8_blockwise on sm103a (#32224)

authored and
felix01.yu
committed
3807f65
[Voxtral Realtime] Introduce global log mel max (#33574)

authored and
felix01.yu
committed
5c2d374
[BugFix] DPMetadata raises assert error for dense model (#32739)

authored and
felix01.yu
committed
e5caed3
[CI/Build] add directions for CPU image upload to Docker Hub (#32032)

authored and
felix01.yu
committed
be84f8c
[torch.compile] Don't do the fast moe cold start optimization if there is speculative decoding (#33624)

authored and
felix01.yu
committed
3bd8bdd
[Feature][CPU Backend]: Optimize ARM vectorization backend (#30329)

Radu2k
authored and
felix01.yu
committed
9bbbac7
[Bugfix] Fix mm budget setting for Qwen Omni models (#33634)

ywang96
authored and
felix01.yu
committed
685a088
[Frontend] Add sampling parameters to Responses API (#32609)

authored and
felix01.yu
committed
6c1d021
Fix quantized Falcon-H1 model loading issues (#32728)

authored and
felix01.yu
committed
6c6e296
[CI/Build] Remove hardcoded America/Los_Angeles timezone from Dockerfiles (#33553)

authored and
felix01.yu
committed
a81c61b
[Bugfix] Interleaved thinking keeps compatibility with reasoning_content (#33635)

authored and
felix01.yu
committed
57f42f6
[XPU][1/N] Deprecate ipex and switch to vllm-xpu-kernels for xpu platform (#33379)

jikunshang
authored and
felix01.yu
committed
1e083c8
[Misc] Remove deprecated profiler environment variables (#33536)

authored and
felix01.yu
committed
9685d2e
[Minor] Some code simplification in `scheduler.py` (#33597)

njhill
authored and
felix01.yu
committed
7ded794
[Misc] Remove deprecated VLLM_ALL2ALL_BACKEND environment variable (#33535)

authored and
felix01.yu
committed
f9c41bf
[torch.compile] Document the workaround to standalone_compile failing (#33571)

zou3519
authored and
felix01.yu
committed
a0427fb
[CI/Build] Investigate torchrun distributed tests hanging issue (#33650)

Isotr0py
authored and
felix01.yu
committed
d8994d8
[Bugfix][Model] Fix DeepSeek-OCR-2 chat template to include BOS token (#33642)

l4b4r4b4b4
authored and
felix01.yu
committed
687b60c
[Refactor] Clean up pooling serial utils (#33665)

DarkLight1337
authored and
felix01.yu
committed
887226f
[Bugfix] Disable RoutingMethodType.[Renormalize,RenormalizeNaive] TRTLLM per-tensor FP8 MoE (#33620)

mgoin
authored and
felix01.yu
committed
e2c023c
[Misc] Update default image format of `encode_base64` (#33656)

DarkLight1337
authored and
felix01.yu
committed
61cf13e
[Bugfix] fix qwen3-asr response error (#33644)

authored and
felix01.yu
committed
8560eb8
Fix offline test for Transformers v5 (#33682)

hmellor
authored and
felix01.yu
committed
fe8e242
Fix Gemma3 GGUF for Transformers v5 (#33683)

hmellor
authored and
felix01.yu
committed
8848c84
[Models] Intern-S1-Pro (#33636)

authored and
felix01.yu
committed
fad0fd5
Fix Gemma3n audio encoder for Transformers v5 (#33673)

hmellor
authored and
felix01.yu
committed
571d57b
Document NixlConnector backend selection via kv_connector_extra_config (#33552)

KrxGu
authored and
felix01.yu
committed
1971cfd
[Bugfix][Async][Connector] avoid vllm-side double free during async scheduling + request abort + async KV cache transfer (#33377)

KuntaiDu
authored and
felix01.yu
committed
8e52e63
Feat/add nemotron nano v3 tests (#33345)

shaharmor98
authored and
felix01.yu
committed
740312b
[Bugfix] Do not add extra \n for image-only cases when constructing multimodal text prompts. (#33647)

noooop
authored and
felix01.yu
committed
813801c
[MM] Pass `prefix` parameter to MMEncoderAttention (#33674)

shen-shanshan
authored and
felix01.yu
committed
4e5b73e
[Voxtral models] Skip warm-up to skip confusing error message in warm-up (#33576)

authored and
felix01.yu
committed
75cbd9e
[Bugfix] Fix startup hang for Granite Speech (#33699)

DarkLight1337
authored and
felix01.yu
committed
b57afa5
[P/D] rework mooncake connector and introduce its bootstrap server (#31034)

authored and
felix01.yu
committed
f1a1cc6
[Attention][FA3] Update FA3 to include new swizzle optimization (#23465)

LucasWilkinson
authored and
felix01.yu
committed
32f9c3d
[torch.compile] Significantly speed up cold start times (#33641)

zou3519
authored and
felix01.yu
committed
b78210a
Turn `@config` into a `dataclass_transform` (#31541)

hmellor
authored and
felix01.yu
committed
53d8492
[MISC] Fix Tensor Parallelism for Quantized Mamba Models with n_groups=1 (#33257)

vadiklyutiy
authored and
felix01.yu
committed
198d8c8
[Voxtral Realtime] Change name (#33716)

patrickvonplaten
authored and
felix01.yu
committed
857742d
[Bugfix] Disable TRTLLM FP8 MoE if router_logits_dtype==float32 and routing_method!=DeepSeekV3 (#33613)

mgoin
authored and
felix01.yu
committed
9b253f6
[Bugfix] Fix sparse MLA metadata building (#33579)

MatthewBonanni
authored and
felix01.yu
committed
34edf82
[Dependency] Remove comments of ray in dependency files (#33351)

yewentao256
authored and
felix01.yu
committed
7b4b2e7
[BugFix][Spec Decoding] Fix negative accepted tokens metric crash (#33729)

njhill
authored and
felix01.yu
committed
a61eb66
[Frontend][4/n] Make pooling entrypoints request schema consensus | ScoreRequest (#33060)

noooop
authored and
felix01.yu
committed
71de4e6
[Bugfix] Fix torchrun PP broadcast deadlock with async scheduling (#33701)

Isotr0py
authored and
felix01.yu
committed
631253b
[1/N] Initial Implementation of Parser for ResponsesAPI (#32712)

authored and
felix01.yu
committed
95f285a
[CPU] Split attention dispatch by head_dim alignment (#32161)

R3hankhan123
authored and
felix01.yu
committed
0865268
[MM] Align the prefix of MMEncoderAttention with Attention (#33750)

shen-shanshan
authored and
felix01.yu
committed
d816bc1
Save startup benchmark results as a list of values (#33629)

huydhn
authored and
felix01.yu
committed
27ef6c3
[Bugfix] Define router_logits_dtype for remaining MoE models (#33737)

mgoin
authored and
felix01.yu
committed
58f0615
[Refactor] Remove unused dead code (#33718)

yewentao256
authored and
felix01.yu
committed
1855e27
[Feature] Enable `TRITON_ATTN` for Batch Invariance (#33688)

frankwang28
authored and
felix01.yu
committed
268fdbe
[Deprecation] Remove `_get_data_parser` in MM processor (#33757)

DarkLight1337
authored and
felix01.yu
committed
dc6325c
[Deprecation] Deprecate profiling envs (#33722)

yewentao256
authored and
felix01.yu
committed
d26cf66
[Hardware][AMD][CI] Refactor AMD tests to properly use BuildKite parallelism (#32745)

mawong-amd
authored and
felix01.yu
committed
6ffc8aa
[Metrics] Add labeled prompt token metrics for P/D disaggregation (#33290)

ZhanqiuHu
authored and
felix01.yu
committed
69f5f82
[XPU] remove common path warning log (#33769)

jikunshang
authored and
felix01.yu
committed
153e356
use ORJSONResponse when available to improve the efficiency of request process (#33548)

staugust
authored and
felix01.yu
committed
dc2a088
[XPU][2/N] add support unquantized moe support for xpu (#33659)

jikunshang
authored and
felix01.yu
committed
b7f62fb
[compile] Clean up AOT compile bypass on evaluate_guards. (#33578)

zhxchen17
authored and
felix01.yu
committed
b8001e9
[compile] Remove runner type from ignored caching factor list. (#33712)

zhxchen17
authored and
felix01.yu
committed
e58c980
[KV Connector][BugFix] scheduler: Delay freeing blocks of aborted async loads (#32255)

orozery
authored and
felix01.yu
committed
fa76d10
[PERF] Change GDN Attention State Layout from [N, HV, K, V] to [N, HV, V, K] (#33291)

vadiklyutiy
authored and
felix01.yu
committed
7bdd641
[Bugfix][Model] Fix audio-in-video support for Qwen2.5-Omni and Qwen3-Omni (#33605)

authored and
felix01.yu
committed
52f5f1b
[Model] Apply #32631 for recent models (#33785)

DarkLight1337
authored and
felix01.yu
committed
24109cc
[Perf] Optimize chat completion streaming performance (#33782)

chaunceyjiang
authored and
felix01.yu
committed
1469dff
Apply #33621 to main (#33758)

authored and
felix01.yu
committed
1a4f2e4
[Bugfix][ROCm] Include float8_e4m3fnuz in NCCL Dtype Dispatching (#33713)

micah-wil
authored and
felix01.yu
committed
88d55db
[Perf] Optimize spec decoding + async scheduling, 1.5% Throughput improvement (#33612)

authored and
felix01.yu
committed
1339352
[Bugfix] Fix `normalize` still being passed to `PoolerConfig` (#33794)

DarkLight1337
authored and
felix01.yu
committed
13b0213
[Bugfix] Fix ubatch wrapper num_tokens calculate (#33694)

jiangkuaixue123
authored and
felix01.yu
committed
27559e1
[Misc] Delay deprecation of CommonAttentionMetadata properties (#33801)

LucasWilkinson
authored and
felix01.yu
committed
8cb07df
[Bugfix] Fix interns1-pro initialization and PP (#33793)

Isotr0py
authored and
felix01.yu
committed
b09d43f
[rocm][ray] Fix: Unify Ray device visibility handling across CUDA and ROCm (#33308)

kouroshHakha
authored and
felix01.yu
committed
ca24900
Implement zero-copy GQA for multimodal and CPU (#33732)

voidbag
authored and
felix01.yu
committed
a9d3c5b
[Bugfix] Support `RotaryEmbedding` CustomOp for gpt-oss (#33800)

simondanielsson
authored and
felix01.yu
committed
76d47eb
[Model] Add transcription support for Qwen3-Omni (#29828)

authored and
felix01.yu
committed
d85d6fb
Revert "[torch.compile] Significantly speed up cold start times" (#33820)

zou3519
authored and
felix01.yu
committed
6c08f2e
Change the type signature of MixtureOfExperts.expert_weights to MutableSequence[Sequence[Tensor]] (#33573)

authored and
felix01.yu
committed
2c4a719
[Core] Don't schedule spec tokens with prefill chunks (#33652)

njhill
authored and
felix01.yu
committed
fe0f1c2
feat: Add ColBERT late interaction model support (#33686)

authored and
felix01.yu
committed
84369fa
[CI][torch.compile] Reduce e2e fusion test time (#33293)

ProExpertProg
authored and
felix01.yu
committed
931e1ec
[Bugfix] Disable TRTLLM attention when KV transfer is enabled (#33192)

ZhanqiuHu
authored and
felix01.yu
committed
b316f7d
[Bugfix] fix DeepSeek R1 with CUTLASS MLA Broken on B200 (#33637)

chaunceyjiang
authored and
felix01.yu
committed
6360163
add inline explaination
felix01.yu
committed
3fdb83d
Merge branch 'vllm-project:main' into patch-1
gameofdimension
authored
b7fba50

Uh oh!

Uh oh!

Commits

Commits on Feb 5, 2026