Optimize MoE Token Dispatch for Tensor Parallel Configurations#22993

Closed

skyloevil wants to merge 293 commits intovllm-project:mainfrom

skyloevil:optimize/moe-dispatch-efficiency

This pull request is big! We're only showing the most recent 250 commits

Commits on Sep 13, 2025

Add @benchislett to codeowner for spec decode and structured outputs (#24362)

benchislett
authored and
skyloevil
committed
[Bugfix] Avoid uninitialized usage of azp_val when AZP is false. (#24335)

mohankku
authored and
skyloevil
committed
[Bugfix] Fix broken deepseek fp8 TP weights loading (#24367)

Isotr0py
authored and
skyloevil
committed
[Bugfix] Fix test_mixtral_moe (#24371)

jeejeelee
authored and
skyloevil
committed
Lora bias(enable_lora_bias) deprecate warning (#24339)

authored and
skyloevil
committed
[Fix] [gpt-oss] fix non-tool calling path for chat completion (#24324)

aarnphm
authored and
skyloevil
committed
[Frontend][Responses API] Support reporting tool output tokens and fix reasoning token count (#24285)

yeqcharlotte
authored and
skyloevil
committed
[Bugfix] Fix unstable silu_mul+nvfp4 quant fusion test (#24370)

elvischenv
authored and
skyloevil
committed
break execute_model in gpu_model_runner into sub-functions for custom scopes (#24265)

bangshengtang
authored and
skyloevil
committed
[V0 deprecation] Deprecate V0 Neuron backend (#21159)

WoosukKwon
authored and
skyloevil
committed
[attention][DCP] use AttentionImpl.need_to_return_lse_for_decode (#24372)

youkaichao
authored and
skyloevil
committed
Migrate Qwen2 inputs to TensorSchema (#23475)

authored and
skyloevil
committed
[CI][Fix] deterministic seed for flaky CI runs on structured outputs (#24380)

aarnphm
authored and
skyloevil
committed
[Benchmark] add benchmark for custom activation op (#23908)

authored and
skyloevil
committed
QWEN3 Thinking Fused MoE kernels Optimization configs (#24330)

samanamp
authored and
skyloevil
committed
[Misc] collect flashinfer version in collect_env.py (#24378)

yeqcharlotte
authored and
skyloevil
committed
[Bugfix] Fix Qwen3-coder moe tuned config (#24072)

jeejeelee
authored and
skyloevil
committed
[TPU] Remove TopKTopPSampler dependency for TPU sampler (#24391)

WoosukKwon
authored and
skyloevil
committed
Add renderer-based prompt processing for embedding and classification endpoints (#24356)

sfeng33
authored and
skyloevil
committed
Skip MM Encoder for non-first PP ranks (#24387)

WoosukKwon
authored and
skyloevil
committed
Add @luccafong to codeowner for spec decode (#24397)

luccafong
authored and
skyloevil
committed
[Kernel] Support decode context parallelism on Blackwell with CUTLASS MLA (#24385)

authored and
skyloevil
committed
[xpu] upgrade ipex/python3.12 for xpu (#23830)

yma11
authored and
skyloevil
committed
[Sampler] Support returning all prompt logprobs (#23868)

authored and
skyloevil
committed
[CI/Build] Disable flaky test_structured_output tests (#24404)

22quinn
authored and
skyloevil
committed
[CI/Build] Fix local image inputs in test_pixtral.py (#24401)

authored and
skyloevil
committed
[Doc] Fix UTF-8 encoding issues in documentation generation on Windows (#24361)

authored and
skyloevil
committed
[P/D] Add a shutdown method to the Connector API (#22699)

chaunceyjiang
authored and
skyloevil
committed
[Model] Remove unnecessary CUDA sync of GLM-4.1V image and video preprocess (#24332)

authored and
skyloevil
committed
[Model] Remove unnecessary CUDA sync of Qwen2VL image and video preprocess (#24334)

authored and
skyloevil
committed
[gpt-oss][Responses API] Fix the function call id format (#24409)

chaunceyjiang
authored and
skyloevil
committed
[Docs] Fix a tip indentation and typo (#24419)

windsonsea
authored and
skyloevil
committed
[Doc]: fix typos in Python comments (#24417)

didier-durand
authored and
skyloevil
committed
[Doc] Fix issues in integrations/llamastack.md (#24428)

windsonsea
authored and
skyloevil
committed
[Bugfix] Fix get_quant_config when using modelscope (#24421)

Potabk
authored and
skyloevil
committed
[Bugfix] Fix mamba2 prefill chunking (#23279)

authored and
skyloevil
committed
[Misc] Terratorch related fixes (#24337)

authored and
skyloevil
committed
Move `KVEventsConfig` from `config/__init__.py` to `config/kv_events.py` (#24433)

hmellor
authored and
skyloevil
committed
[Frontend] User-provided uuids for medias in chat. (RFC #22044) (#23449)

authored and
skyloevil
committed
[Docs] Move feature compatibility tables to README (#24431)

hmellor
authored and
skyloevil
committed
[Doc]: fix 2 hyperlinks leading to Ray site after they changed Ray's doc structure (#24438)

authored and
skyloevil
committed
[Docs]add eplb_config param use docs (#24213)

lengrongfu
authored and
skyloevil
committed
[Model] Enable BNB support for qwen2_5_omni_thinker (#24420)

jeejeelee
authored and
skyloevil
committed
[Spec Decode][Benchmark] Add Spec Bench Dataset for benchmarking (#23563)

ekagra-ranjan
authored and
skyloevil
committed
[Spec Decode][Benchmark] Add Blitzedit dataset (#23605)

authored and
skyloevil
committed
[Model] Remove quantized mixtral (#24437)

jeejeelee
authored and
skyloevil
committed
[CI] Enable encoder model compilation test (#24442)

ZJY0516
authored and
skyloevil
committed
[Model loader]: support multi-thread model weight loading (#23928)

authored and
skyloevil
committed
[Spec Decode] Fix offline spec_decode.py (#24257)

authored and
skyloevil
committed
[Attention] FlashAttention MLA cudagraph support (#23958)

authored and
skyloevil
committed
[Bugfix] Disable the statslogger if the api_server_count is greater than 1 (#22227)

authored and
skyloevil
committed
[Hardware][IBM Z] Fix Outlines Core issue for s390x (#24034)

R3hankhan123
authored and
skyloevil
committed
[CI] Add nightly multiarch manifests to dockerhub (#24102)

authored and
skyloevil
committed
Update reviewers for modelopt related files (#24468)

Edwardf0t1
authored and
skyloevil
committed
[Bugfix][Wide EP] Fix redundant work when using DeepEP, TP Attn, and EP MoE (#24134)

tlrmchlsmth
authored and
skyloevil
committed
[gpt-oss] Harmony changes with container tool support (#23386)

authored and
skyloevil
committed
Bump actions/setup-python from 5.4.0 to 6.0.0 (#24414)

authored and
skyloevil
committed
[doc] update `vllm serve` cli args documentation (#24329)

cjackal
authored and
skyloevil
committed
Bump actions/stale from 9.1.0 to 10.0.0 (#24412)

authored and
skyloevil
committed
Bump actions/github-script from 7.0.1 to 8.0.0 (#24413)

authored and
skyloevil
committed
Move `KVTransferConfig` from `config/__init__.py` to `config/kv_transfer.py` (#24434)

hmellor
authored and
skyloevil
committed
[BugFix][Model] Fix Ernie4.5-VL hanging on long inputs (#24074)

CSWYF3634076
authored and
skyloevil
committed
[Flashinfer] Support Flashinfer TRTLLM FP8-qkv BF16/FP16-out Attention Kernel (#23647)

elvischenv
authored and
skyloevil
committed
[Core] Use sha256 bytes instead of BlockHash to reduce GC overhead (#23673)

linzebing
authored and
skyloevil
committed
Add data_parallel_size to VllmConfig string representation (#24298)

authored and
skyloevil
committed
[Bugfix] Fix Apertus HF repo name (#24447)

DarkLight1337
authored and
skyloevil
committed
[Misc] Improve Worker process title and logging prefix (#22205)

22quinn
authored and
skyloevil
committed
[Doc] mention fpdb for multiprocess breakpoints (#24452)

mickaelseznec
authored and
skyloevil
committed
[Misc] Support bench serve long context (#24373)

minosfuture
authored and
skyloevil
committed
[Doc]: fixing typos to improve docs (#24480)

didier-durand
authored and
skyloevil
committed
[Performance][MM] Building the inverse permutation in O(n) time in Qwen2_5_VisionTransformer (#24443)

authored and
skyloevil
committed
[Misc] Add claude settings to gitignore (#24492)

yeqcharlotte
authored and
skyloevil
committed
[Misc] Add Codex settings to gitignore (#24493)

authored and
skyloevil
committed
[gpt-oss] Validate gpt-oss python tool during initialization (#23856)

heheda12345
authored and
skyloevil
committed
[RL] fast weight update with zmq + ipc handles (#24295)

authored and
skyloevil
committed
[CI/Build][Doc] Fully deprecate old bench scripts for serving / throughput / latency (#24411)

yeqcharlotte
authored and
skyloevil
committed
[Compilation][WideEP] Enable Piecewise CUDAGraph for DeepEPHT (#24123)

yewentao256
authored and
skyloevil
committed
[Model] Systematic support for fp32 head, pooling models part (#23810)

noooop
authored and
skyloevil
committed
[Bugfix] Handle the edge case in detokenizer where processed tokens contain both `stop` str and `eos` token (#23938)

dtransposed
authored and
skyloevil
committed
[Core] Run garbage collector after CUDA graph capture to fix throughput regression (#24128)

authored and
skyloevil
committed
[Kernels] Add Flash Linear Attention Kernels (#24518)

youkaichao
authored and
skyloevil
committed
[ROCm][CI/Build] Sync ROCm dockerfiles with the ROCm fork (#24279)

gshtras
authored and
skyloevil
committed
[Bugfix] Fix hidden_size for multimodal classification model (#24501)

jeejeelee
authored and
skyloevil
committed
Extend renderer with embedding support and integrate completion endpoint (#24405)

sfeng33
authored and
skyloevil
committed
[Misc] bump outlines_core to fix the version conflicts with outlines >= 1.2.0 (#24368)

authored and
skyloevil
committed
[Docs] Gemma3n `transcriptions` endpoint support (#24512)

NickLucche
authored and
skyloevil
committed
[TPU] Fix tpu structured decoding in mixed batches (#24458)

Chenyaaang
authored and
skyloevil
committed
[CI] execute all piecewise compilation tests together (#24502)

ZJY0516
authored and
skyloevil
committed
[Feature] Disallow FlashMLA on Blackwell (#24521)

authored and
skyloevil
committed
[Log] Use a relative path in debug-level logs to distinguish files with identical names (#23846)

ZJY0516
authored and
skyloevil
committed
[Benchmark] Update bench doc with mtbench, blazedit, spec bench (#24450)

ekagra-ranjan
authored and
skyloevil
committed
[Benchmark] Add option to skip oversampling in benchmark (#24457)

ekagra-ranjan
authored and
skyloevil
committed
[ROCm][Feature] Enable Pipeline Parallelism with Ray Compiled Graph on ROCm (#24275)

charlifu
authored and
skyloevil
committed
[Bugfix] Improve EPLB config validation error message (#24524)

tlrmchlsmth
authored and
skyloevil
committed
[Bugfix] Fix for 24530. Fix naive all2all shared expert overlap. (#24538)

bnellnm
authored and
skyloevil
committed
[Perf] Convert np array to torch tensor to index into block table for attn chunking (#24474)

sarckk
authored and
skyloevil
committed
Add @heheda12345 to CODEOWNERS of KVCacheManager related code (#24546)

heheda12345
authored and
skyloevil
committed
[CI] Retry flaky fp8 cutlass mla tests (#24536)

njhill
authored and
skyloevil
committed
[Hardware][Apple-CPU] Enable native bfloat16 on Apple Silicon (M2 and later) (#24129)

ignaciosica
authored and
skyloevil
committed
[BugFix] Fix async core engine client finalizer (#24540)

njhill
authored and
skyloevil
committed
[CI] Adjust threshold for flaky ngram spec decoding test (#24528)

njhill
authored and
skyloevil
committed
[KV Connector] More async support for `get_num_new_matched_tokens` (#23620)

ApostaC
authored and
skyloevil
committed
[P/D] MultiConnector supports shutdown (#24425)

chaunceyjiang
authored and
skyloevil
committed
[BugFix][Spec Decode] Fix out-of-range index triggered by eagle3; re-enable test for LlamaForCausalLMEagle3 (#24392)

wwl2755
authored and
skyloevil
committed
[gpt-oss] Cache permute indices for faster MXFP4 MoE layer loading (#24154)

frank-wei
authored and
skyloevil
committed
[Core] Simplify and unify mm uuid handling & auto-generated mm hash overrides processing. (#24271)

huachenheli
authored and
skyloevil
committed
[Bugfix] Update Run:AI Model Streamer Loading Integration (#23845)

authored and
skyloevil
committed
[Docs] Enable relative links in examples to function when rendered in the docs (#24041)

hmellor
authored and
skyloevil
committed
[docs] promo pytorch conf and ray summit (#24562)

simon-mo
authored and
skyloevil
committed
[Bugfix] Guard `_may_reorder_batch` for encoder-only models on CPU (#24319) (#24348)

authored and
skyloevil
committed
Consolidate rendering parameters into RenderConfig dataclass (#24543)

sfeng33
authored and
skyloevil
committed
[Model] Limit CPU threads for image transformations in InternVL to reduce cpu contention. (#24519)

authored and
skyloevil
committed
[Attention] add DCP support for FLASH_ATTN_MLA backend (#24453)

authored and
skyloevil
committed
[ROCm][Bugfix] Fix Aiter RMSNorm (#23412)

vllmellm
authored and
skyloevil
committed
[Docs] Improve organisation of API Reference nav (#24569)

hmellor
authored and
skyloevil
committed
[Docs] Document the extra memory footprint overhead when using EPLB (#24537)

tlrmchlsmth
authored and
skyloevil
committed
Support for NemotronH Nano VLM (#23644)

danielafrimi
authored and
skyloevil
committed
Feature/vit attention unification# 23880 (#23978)

authored and
skyloevil
committed
[LoRA]: Add LoRA support to Mistral's Voxtral models (#24517)

authored and
skyloevil
committed
Move `LoadConfig` from `config/__init__.py` to `config/load.py` (#24566)

hmellor
authored and
skyloevil
committed
[BugFix][Multi Modal] Fix TensorSchema shape mismatch in Molmo (#24559)

wwl2755
authored and
skyloevil
committed
[BugFix][easy] Fix flaky test test_gpt_oss_multi_turn_chat (#24549)

authored and
skyloevil
committed
[BugFix] Ensure integrity of reused CPU tensors during async scheduling (#24527)

authored and
skyloevil
committed
[CI/Build] split true unit tests to Entrypoints Unit Tests (#24418)

yeqcharlotte
authored and
skyloevil
committed
[rocm] enable torchao quantization for rocm (#24400)

draftbk
authored and
skyloevil
committed
[CI] Add PPL test for generation models (#24485)

noooop
authored and
skyloevil
committed
[CI/Build] bump timm dependency (#24189)

authored and
skyloevil
committed
fix some typos (#24167)

authored and
skyloevil
committed
Fix Auto_Round Quatization Loading on SM75 and Lower GPUs (#24217)

authored and
skyloevil
committed
[Docs] Fix warnings in `mkdocs build` (continued) (#24092)

authored and
skyloevil
committed
[BugFix] `python collect_env.py` and `vllm collect-env` compatibility with uv venv (#24066)

yankay
authored and
skyloevil
committed
[Platform] Custom ops support for LMhead and LogitsProcessor (#23564)

zzhx1
authored and
skyloevil
committed
[CI] Fix tensorizer test assertion (#24545)

pwschuurman
authored and
skyloevil
committed
[Core] Split LoRA layers (#24574)

jeejeelee
authored and
skyloevil
committed
[Doc] Add documentation for GLM-4.5 series models: tool-calling and reasoning parser (#24589)

WangErXiao
authored and
skyloevil
committed
[Logging] allow config logging stream (#24336)

842974287
authored and
skyloevil
committed
[Bugfix] fix modelopt exclude_modules name mapping (#24178)

authored and
skyloevil
committed
[Bugfix] Fix DeepEP config for DP4TP4 (#23619)

minosfuture
authored and
skyloevil
committed
[Core] Support configuration parsing plugin (#24277)

authored and
skyloevil
committed
[Misc] update log level debug to warning when process port is used by (#24226)

authored and
skyloevil
committed
[Bugfix] Enable FP8 KV cache for FlashInfer and Triton backend on non-sm100 GPUs (#24577)

gau-nernst
authored and
skyloevil
committed
[CI] Fail subprocess tests with root-cause error (#23795)

njhill
authored and
skyloevil
committed
[v1] Add Whisper model support (encoder-decoder) (#21088)

authored and
skyloevil
committed
[torch.compile][ROCm][V1] Enable attention output FP8 fusion for V1 attention backends (#19767)

authored and
skyloevil
committed
[gpt-oss] raise error for flashinfer backend without trtllm (#24482)

heheda12345
authored and
skyloevil
committed
[Perf] Warmup FlashInfer attention during startup (#23439)

authored and
skyloevil
committed
[Kernel] Flashinfer MLA (trtllm-gen) decode kernel integration (#21078)

authored and
skyloevil
committed
[Misc] Make timeout passable in init_distributed_environment (#24522)

jberkhahn
authored and
skyloevil
committed
[Models][Quantization] Add quantization configuration update in Voxtral model (#24122)

authored and
skyloevil
committed
[distributed] update known issues (#24624)

youkaichao
authored and
skyloevil
committed
Add @chaunceyjiang to codeowner for reasoning Reasoning and Tool parser (#24406)

chaunceyjiang
authored and
skyloevil
committed
[Bug] [Spec Decode] Fix model_initialization test and mismatch in aux_hidden_layers (#24613)

authored and
skyloevil
committed
[Ultravox] Fix Gemma instantiation, support quantization via --hf-overrides (#24131)

petersalas
authored and
skyloevil
committed
[Bugfix] Add missing VIT backend dispatch on CPU (#24623)

bigPYJ1151
authored and
skyloevil
committed
[BugFix] Fix pipeline parallel (#24621)

njhill
authored and
skyloevil
committed
[Engine][Chore] use local variable and remove output var assignment (#24554)

GuyStone
authored and
skyloevil
committed
Kimi K2 Fused MoE kernels Optimization configs (#24597)

samanamp
authored and
skyloevil
committed
Enable --profile in 'vllm bench throughput' (#24575)

tomasruizt
authored and
skyloevil
committed
[Core] feat: Add --safetensors-load-strategy flag for faster safetensors loading from Lustre (#24469)

authored and
skyloevil
committed
[Doc]: fixing doc typos (#24635)

didier-durand
authored and
skyloevil
committed
[Model] New model support for Motif-1-Tiny (#23414)

authored and
skyloevil
committed
Remove redundant all gather + split (#23441)

authored and
skyloevil
committed
[torchao] Support quantization configs using module swap (#21982)

jerryzh168
authored and
skyloevil
committed
Add the support for the qwen3 next model (a hybrid attention model). (#24526)

authored and
skyloevil
committed
[Bugfix] Fix incorrect import of CacheConfig (#24631)

DarkLight1337
authored and
skyloevil
committed
[Docs] Revise frameworks/anything-llm.md (#24489)

windsonsea
authored and
skyloevil
committed
[Docs] Update V1 doc to reflect whisper support (#24606)

russellb
authored and
skyloevil
committed
[Docs] Use 1-2-3 list for deploy steps in deployment/frameworks/ (#24633)

windsonsea
authored and
skyloevil
committed
[CI]Add transformers_utils to Async Engine, Inputs, Utils, Worker Test (#24615)

charlotte12l
authored and
skyloevil
committed
[Bugfix] Fix _synced_weight_loader (#24565)

kyuyeunk
authored and
skyloevil
committed
[CI] Split pooling from entrypoints Test (#24632)

noooop
authored and
skyloevil
committed
[Misc] Add @NickLucche to codeowners (#24647)

NickLucche
authored and
skyloevil
committed
[CI Failure] fix models/language/pooling/test_auto_prefix_cache_support.py (#24636)

noooop
authored and
skyloevil
committed
Fix typing for `safetensors_load_strategy` (#24641)

hmellor
authored and
skyloevil
committed
Move `LoRAConfig` from `config/__init__.py` to `config/lora.py` (#24644)

hmellor
authored and
skyloevil
committed
[XPU] add missing dependency tblib for XPU CI (#24639)

faaany
authored and
skyloevil
committed
[Docs] Fixes a typo in the qwen3next model name. (#24654)

sighingnow
authored and
skyloevil
committed
[build] add torch to tool.uv no-build-isolation-package (#24303)

authored and
skyloevil
committed
[Bench] Add qwen-next in benchmark_moe.py (#24661)

jeejeelee
authored and
skyloevil
committed
[CI] Split mteb test from Language Models Test (#24634)

noooop
authored and
skyloevil
committed
Allow users to specify kv cache memory size (#21489)

authored and
skyloevil
committed
[HybridKVCache][Platform] Add support_hybrid_kv_cache for platform (#24646)

MengqingCao
authored and
skyloevil
committed
[Bugifx] Fix qwen-next packed_modules_mapping (#24656)

jeejeelee
authored and
skyloevil
committed
[Docs] Add transcription support to model (#24664)

NickLucche
authored and
skyloevil
committed
[Doc] Fix Markdown Pre-commit Error (#24670)

yewentao256
authored and
skyloevil
committed
[Docs] Fix typos in EP deployment doc (#24669)

hmellor
authored and
skyloevil
committed
[VLM] Optimize GLM4.5-V-style video processing to only decode necessary frames (#24161)

Isotr0py
authored and
skyloevil
committed
[Kernels] Enable Torch Symmetric Memory All-Reduce By Default (#24111)

authored and
skyloevil
committed
[Bugfix] Fix platform-specific routing in CustomOp implementations (#24444)

kzawora-intel
authored and
skyloevil
committed
Fix model name included in responses (#24663)

hmellor
authored and
skyloevil
committed
fix some typos (#24616)

co63oc
authored and
skyloevil
committed
[Docs] Fix formatting of transcription doc (#24676)

hmellor
authored and
skyloevil
committed
[VLM] Migrate remain DP-supported ViT models to use `disable_tp` (#24363)

Isotr0py
authored and
skyloevil
committed
[Ultravox] Use wrapped_model_config to instantiate inner model (#24679)

petersalas
authored and
skyloevil
committed
[Doc] Remove Useless Comments (#24687)

yewentao256
authored and
skyloevil
committed
[Qwen3-Next] Add MoE Config for H200 (#24688)

WoosukKwon
authored and
skyloevil
committed
[BugFix] Fix tokenize asyncio task leak (#24677)

njhill
authored and
skyloevil
committed
Update Spec Decode metrics to include drafted and accepted token throughput (#24127)

qandrew
authored and
skyloevil
committed
[Kernel][B200] `mxfp4` fused cutlass moe (#23696)

authored and
skyloevil
committed
[flashinfer] [kernel] support for fp8 kv cache for trtllm prefill attention (#24197)

mxz297
authored and
skyloevil
committed
[Bugfix] Set `VLLM_ALLREDUCE_USE_SYMM_MEM` default to False (#24696)

yewentao256
authored and
skyloevil
committed
[Qwen3-Next] MoE configs for H200 TP=1,2,4 (#24695)

WoosukKwon
authored and
skyloevil
committed
[CI/Build] Add bc-linter to vLLM CI (#21234)

zhewenl
authored and
skyloevil
committed
[Qwen3-Next] Add B200 MoE configs for Qwen3-next (#24698)

vadiklyutiy
authored and
skyloevil
committed
[Bugfix][Attention] Fix FlashInfer MLA block size logic (#24692)

MatthewBonanni
authored and
skyloevil
committed
[Perf] Use upstream CUTLASS for SM90 Block FP8 kernel (#23280)

authored and
skyloevil
committed
[Qwen3-Next] MOE configs for H100 TP4 (#24699)

heheda12345
authored and
skyloevil
committed
[Doc] Clarify cudagraph capture size logic and default behavior in scheduler (#18698)

authored and
skyloevil
committed
[Bug] Fix Layer `weight_block_size` Assertion Issue (#24674)

yewentao256
authored and
skyloevil
committed
[Startup] Make DeepGEMM warmup scale with max-num-batched-tokens (#24693)

LucasWilkinson
authored and
skyloevil
committed
[V1] feat:add engine v1 tracing (#20372)

authored and
skyloevil
committed
[Bugfix] fixes the causal_conv1d_update kernel update non-speculative decoding cases (#24680)

authored and
skyloevil
committed
[Qwen3-Next] MoE configs for H20 TP=1,2,4,8 (#24707)

jeejeelee
authored and
skyloevil
committed
[DOCs] Update ROCm installation docs section (#24691)

gshtras
authored and
skyloevil
committed
Enable conversion of multimodal models to pooling tasks (#24451)

maxdebayser
authored and
skyloevil
committed
Fix implementation divergence for BLOOM models between vLLM and HuggingFace when using prompt embeds (#24686)

qthequartermasterman
authored and
skyloevil
committed
[Bugfix] Fix MRoPE dispatch on CPU (#24712)

bigPYJ1151
authored and
skyloevil
committed
[BugFix] Fix Qwen3-Next PP (#24709)

njhill
authored and
skyloevil
committed
[CI] Fix flaky test v1/worker/test_gpu_model_runner.py::test_kv_cache_stride_order (#24640)

heheda12345
authored and
skyloevil
committed
[CI] Add ci_envs for convenient local testing (#24630)

noooop
authored and
skyloevil
committed
[CI/Build] Skip prompt embeddings tests on V1-only CPU backend (#24721)

bigPYJ1151
authored and
skyloevil
committed
[Misc][gpt-oss] Add gpt-oss label to PRs that mention harmony or related to builtin tool call (#24717)

heheda12345
authored and
skyloevil
committed
[Bugfix] Fix BNB name match (#24735)

jeejeelee
authored and
skyloevil
committed
[Kernel] [CPU] refactor `cpu_attn.py:_run_sdpa_forward` for better memory access (#24701)

ignaciosica
authored and
skyloevil
committed
[sleep mode] save memory for on-the-fly quantization (#24731)

youkaichao
authored and
skyloevil
committed
[Multi Modal] Add FA3 in VIT (#24347)

wwl2755
authored and
skyloevil
committed
[Multimodal] Remove legacy multimodal fields in favor of MultiModalFeatureSpec (#24548)

sfeng33
authored and
skyloevil
committed
[Doc]: fix typos in various files (#24726)

didier-durand
authored and
skyloevil
committed
[Docs] Fix warnings in mkdocs build (continued) (#24740)

Zerohertz
authored and
skyloevil
committed
[Bugfix] Fix MRoPE dispatch on XPU (#24724)

yma11
authored and
skyloevil
committed
[Qwen3-Next] MoE configs for H100 TP=1,2 and TP2/EP (#24739)

elvircrn
authored and
skyloevil
committed
[Core] Shared memory based object store for Multimodal data caching and IPC (#20452)

dongluw
authored and
skyloevil
committed
[Bugfix][Frontend] Fix `--enable-log-outputs` does not match the documentation (#24626)

kebe7jun
authored and
skyloevil
committed
[Models] Optimise and simplify `_validate_and_reshape_mm_tensor` (#24742)

lgeiger
authored and
skyloevil
committed
[Models] Prevent CUDA sync in Qwen2.5-VL (#24741)

lgeiger
authored and
skyloevil
committed
[Model] Switch to Fused RMSNorm in GLM-4.1V model (#24733)

SamitHuang
authored and
skyloevil
committed
[UX] Remove AsyncLLM torch profiler disabled log (#24609)

mgoin
authored and
skyloevil
committed
[CI] Speed up model unit tests in CI (#24253)

afeldman-nm
authored and
skyloevil
committed
[Bugfix] Fix incompatibility between #20452 and #24548 (#24754)

DarkLight1337
authored and
skyloevil
committed
[CI] Trigger BC Linter when labels are added/removed (#24767)

zhewenl
authored and
skyloevil
committed
[Benchmark] Allow arbitrary headers to be passed to benchmarked endpoints (#23937)

smarterclayton
authored and
skyloevil
committed
[Compilation Bug] Fix Inductor Graph Output with Shape Issue (#24772)

yewentao256
authored and
skyloevil
committed
Invert pattern order to make sure that out_proj layers are identified (#24781)

anmarques
authored and
skyloevil
committed
[Attention][FlashInfer] Enable FP8 FlashInfer (TRTLLM) MLA decode (#24705)

MatthewBonanni
authored and
skyloevil
committed
Add FLASHINFER_MLA to backend selector test (#24753)

MatthewBonanni
authored and
skyloevil
committed
[Qwen3Next] Fixes the cuda graph capture conditions under large batch sizes (#24660) (#24667)

sighingnow
authored and
skyloevil
committed
[Core] Support async scheduling with uniproc executor (#24219)

authored and
skyloevil
committed
[Frontend][Multimodal] Allow skipping media data when UUIDs are provided. (#23950)

authored and
skyloevil
committed
Resolve rebase conflict in batched_deep_gemm_moe: use _get_effective_num_dispatchers() for workspace sizing
skyloevil
committed
Merge branch 'main' into optimize/moe-dispatch-efficiency
skyloevil
authored

Uh oh!

Commits

Commits on Sep 13, 2025