[lora/moe] Improve fused MoE‑LoRA kernel indexing and memory access#32770

Merged

vllm-bot merged 137 commits intovllm-project:mainfrom

cwazai:pr1/moe-lora-kernel-opt

Jan 26, 2026

Commits on Jan 25, 2026

Improve fused MoE‑LoRA kernel indexing and memory access
cwazai
committed
Remove redundant mask in expert_id_i32 load
cwazai
committed
Modifications to remove irrelevant optimizations
cwazai
committed
Kernel (_fused_moe_lora_kernel):
cwazai
committed
Remove split_k
cwazai
committed
1. Rename MAX_LORAS_TOTAL to max_loras for clarity
cwazai
committed
resolve conflict
cwazai
committed
[Bugfix] Force using spawn multiprocess method when it's the WSL platform (#32749)

jasonyanwenl
authored and
cwazai
committed
[Model] Add Eagle2.5-8B Vision-Language Model support (#32456)

George-Polya
authored and
cwazai
committed
[bugfix] Aria model (#32727)

divakar-amd
authored and
cwazai
committed
[MoE Refactor] Oracle Select FP8+NVFP4 Kernels In Priority (#32414)

robertgshaw2-redhat
authored and
cwazai
committed
[Quantization][Deprecation] Remove `DeepSpeedFp8` (#32679)

authored and
cwazai
committed
[Quantization][Deprecation] Deprecate HQQ (#32681)

authored and
cwazai
committed
[ROCm][Deepseekv3.2] Refactor Sparse Indexer as CustomOp (#29287)

ganyi1996ppo
authored and
cwazai
committed
[Quantization][Deprecation] Remove RTN (#32697)

authored and
cwazai
committed
[PluggableLayer][1/N] Define PluggableLayer (Fix ci) (#32744)

whx-sjtu
authored and
cwazai
committed
Bump Flashinfer to v0.6.1 (#30993)

elvischenv
authored and
cwazai
committed
[Misc] Omit "disable NCCL for DP sync" startup log when not applicable (#32707)

njhill
authored and
cwazai
committed
[Model Runner V2] Minor refactor for `compute_slot_mappings` (#32794)

WoosukKwon
authored and
cwazai
committed
Add missing import of fused_topk to benchmark_moe (#32784)

danisereb
authored and
cwazai
committed
[ROCm] fix import for on_gfx9 (#32783)

divakar-amd
authored and
cwazai
committed
[ModelRunner V2] Don't pin reused flashinfer tensors (#32799)

njhill
authored and
cwazai
committed
[Misc] Add Helion version check to collect_env (#32797)

gmagogsfm
authored and
cwazai
committed
[Kernel] Add topk_sigmoid kernel (#31246)

xyang16
authored and
cwazai
committed
[Model Runner V2] Refactor Prompt Logprobs (#32811)

WoosukKwon
authored and
cwazai
committed
[Model Runner V2] Do not error on attention backends (#32820)

WoosukKwon
authored and
cwazai
committed
[Deprecation] Remove deprecated environment variables (#32812)

yewentao256
authored and
cwazai
committed
[Bugfix] Fix potential EAGLE spec decode segfault during graph capture (#32818)

mawong-amd
authored and
cwazai
committed
[EC Connector] Optimize remote cache check in scheduler (#32585)

knlnguyen1802
authored and
cwazai
committed
Cleanup some huggingface_hub-related stuff (#32788)

Wauplin
authored and
cwazai
committed
[Docs] Remove outdated async_scheduling limitation with speculative decoding (#32775)

ikaadil
authored and
cwazai
committed
[ROCm][CI] fix get_valid_backends (#32787)

divakar-amd
authored and
cwazai
committed
[FlashMLA] Update FlashMLA to expose new arguments (#32810)

LucasWilkinson
authored and
cwazai
committed
[Llama.py -> mistral.py] Extract mistral-only relevant code into separate file (#32780)

patrickvonplaten
authored and
cwazai
committed
Upgrade transformers-4.57.5 (#32287)

huydhn
authored and
cwazai
committed
[ROCm][CI] Lower Acceptance Len Threshold For test_draft_model_quantization (#32731)

micah-wil
authored and
cwazai
committed
[ROCm][CI] Fix AITER test flakiness by using explicit attention backend (#32346)

authored and
cwazai
committed
[ROCm][CI][Docs] Add comment explaining TRITON_ATTN fallback for ROCm (#32835)

AndreasKaratzas
authored and
cwazai
committed
[bench] add start_times field to vllm bench serve json result (#32667)

kebe7jun
authored and
cwazai
committed
[Model] Extend `collect_children` and `no_init_weights` contexts (#32757)

DarkLight1337
authored and
cwazai
committed
[AMD][ROCm] MoRI EP: a high-performance all2all backend (#28664)

alexsun07
authored and
cwazai
committed
[Misc] Replace urllib's `urlparse` with urllib3's `parse_url` (#32746)

Isotr0py
authored and
cwazai
committed
[Benchmark] Don't default to `temperature==0` in `vllm bench serve` (#32723)

njhill
authored and
cwazai
committed
Enable Cross layers KV cache layout at NIXL Connector (#30207)

authored and
cwazai
committed
[Frontend][2/n] Make pooling entrypoints request schema consensus | ChatRequest (#32574)

noooop
authored and
cwazai
committed
[Bugfix] Fix Whisper/encoder-decoder GPU memory leak (#32789)

NickLucche
authored and
cwazai
committed
[CI] refactor release pipeline config into groups (#32833)

Harry-Chen
authored and
cwazai
committed
[Frontend] add prompt_cache_key for openresponses (#32824)

chaunceyjiang
authored and
cwazai
committed
OffloadingConnector: Support kernel_block_size != block_size (#30692)

orozery
authored and
cwazai
committed
[Frontend] Introduce Renderer for processing chat messages (using `ModelConfig`) (#30200)

DarkLight1337
authored and
cwazai
committed
[Misc][BE] Turn on strict type coverage for vllm/compilation (#31756)

Lucaskabela
authored and
cwazai
committed
[torch.compile] Improve Cold Start for MoEs (#32805)

zou3519
authored and
cwazai
committed
[Cleanup] Move scheduler `get_routed_experts` logic to separate method (#32706)

njhill
authored and
cwazai
committed
[Misc] Bump opencv-python dependecy version to 4.13 (#32668)

Isotr0py
authored and
cwazai
committed
Support bge-m3 sparse embeddings and colbert embeddings (#14526)

maxdebayser
authored and
cwazai
committed
[Bugfix] ModelScope is supported when downloading LORA models. (#32844)

AuYang261
authored and
cwazai
committed
[Hardware][AMD][CI][Bugfix] Fix regressions from deprecated env vars (#32837)

mawong-amd
authored and
cwazai
committed
[MISC] Add .cursor to .gitignore (#32868)

vadiklyutiy
authored and
cwazai
committed
[UX] Default api_server_count to dp_size if not specified (#32525)

tlrmchlsmth
authored and
cwazai
committed
Support custom URI schemes and trace handlers for profiler (#32393)

diviramon
authored and
cwazai
committed
[Feature] Add --ssl-ciphers CLI argument for TLS cipher control (#30937)

ricky-chaoju
authored and
cwazai
committed
[CI][Attention] Add more CI dependencies for attention tests (#32487)

MatthewBonanni
authored and
cwazai
committed
[CPU Backend] [Perf] Accelerate tensor-parallel/data-parallel inference across NUMA domains on Arm (#32792)

fadara01
authored and
cwazai
committed
[Bugfix][Attention] Explicitly report support for kv_cache_dtype bfloat16 (#32795)

MatthewBonanni
authored and
cwazai
committed
Add llmcompressor fp8 kv-cache quant (per-tensor and per-attn_head) (#30141)

eldarkurtic
authored and
cwazai
committed
[Refactor] Remove unused tpu files (#32610)

yewentao256
authored and
cwazai
committed
[Perf] Create TMA-aligned input scale tensor for DeepGemm on Hopper (#32619)

xyang16
authored and
cwazai
committed
[BugFix] Fix invalid flashinfer_fused_moe_blockscale_fp8 op registration (#32855)

fadara01
authored and
cwazai
committed
[MoE Refactor] Move `select_experts` from `FusedMoEQuantMethod` -> `FusedMoE` (#31996)

bnellnm
authored and
cwazai
committed
[Misc] Log vLLM logo when starting server (#32796)

njhill
authored and
cwazai
committed
[BugFix] deepseek_v32_encoding: Replace asserts with proper exceptions (#32884)

RishabhSaini
authored and
cwazai
committed
[torch.compile] Compile `CustomOp.forward_native` for `SiluAndMul` and `QuantFP8` to avoid raw torch ops inside opaque custom ops (#32806)

authored and
cwazai
committed
[CI] Fix mypy for `vllm/v1/structured_output` (#32722)

yewentao256
authored and
cwazai
committed
[Bugfix] Fix _CPU_MOE_ACT AssertionError when vLLM config not set (#32777)

karanb192
authored and
cwazai
committed
[CI][Models] Add VLM Support for Sequence Classification Conversion (#32885)

AndreasKaratzas
authored and
cwazai
committed
[Misc] Add `get_name` to missing AttentionBackends (#32698)

NickLucche
authored and
cwazai
committed
[CI/Build][CPU] Fix failed pooling tests and macos smoke test (#32907)

authored and
cwazai
committed
[Voxtral] Add new streaming arch (#32861)

authored and
cwazai
committed
[Frontend][3/n] Make pooling entrypoints request schema consensus | EmbedRequest & ClassifyRequest (#32905)

authored and
cwazai
committed
[CPU Backend][BugFix] Fix failing CPU MoE test (#32876)

fadara01
authored and
cwazai
committed
[Benchmark][Bugfix] Fix race condtion when starting server for sweep benchmark (#32927)

Isotr0py
authored and
cwazai
committed
[CPU][Feat] Update PyTorch to v2.10 for CPU Backend (#32869)

authored and
cwazai
committed
[Feature]: Remove DtoH Copy for lfm2_vl On Default Stream (#32815)

tianshu-Michael-yu
authored and
cwazai
committed
[Bugfix] Fix getting vision features in Transformer Multimodal backend (#32933)

zucchini-nlp
authored and
cwazai
committed
[Bugfix] Disable tma_aligned_scales in test_fusions_e2e (#32916)

xyang16
authored and
cwazai
committed
[Misc] Postpone torch_profiler deprecation (#32867)

NickLucche
authored and
cwazai
committed
[Bugfix] Fix FP8 MoE EP Weight Loading for ModelOpt Llama4 (#32886)

baonudesifeizhai
authored and
cwazai
committed
[ROCm][PD] Remove unused moriio connector proxy code (#32939)

markmc
authored and
cwazai
committed
[Hardware][AMD][CI][Bugfix] Fix Kernels Attention Cache test (#32904)

mawong-amd
authored and
cwazai
committed
[Frontend] add logprob, compression_rate to 'verbose_json' features (#31059)

sangbumlikeagod
authored and
cwazai
committed
[torch.compile][CI] Add back attn fusion on hopper/ada (#32940)

ProExpertProg
authored and
cwazai
committed
[Model] Enable LoRA support for internvl2 (#32397)

MatteoFari
authored and
cwazai
committed
[V1][Hybrid] Mamba Prefix Caching with align mode (#30877)

authored and
cwazai
committed
[CI][torch nightlies] Use main Dockerfile with flags for nightly torch tests (#30443)

authored and
cwazai
committed
[Bugfix][CI] Fix pre-commit (#32956)

MatthewBonanni
authored and
cwazai
committed
[Model Runner V2] Add KV Connector support (#32742)

njhill
authored and
cwazai
committed
[cudagraphs] Refactor cudagraph capture loop (#32946)

LucasWilkinson
authored and
cwazai
committed
fix: Add glm4_moe_lite to MLA detection (#32614)

authored and
cwazai
committed
[Bug] Fix benchmark script `moe_permute_unpermute` (#32949)

yewentao256
authored and
cwazai
committed
[CI][AMD][BugFix] Update wvSplitK (and other skinny_gemm wrappers) to ensure tensors passed will be made contiguous for the kernel (#32831)

authored and
cwazai
committed
[Refactor] Rename `gptq_marlin` to `marlin` to match MoE (#32952)

mgoin
authored and
cwazai
committed
[Refactor] Clean up unused variables & func (#32692)

yewentao256
authored and
cwazai
committed
[Bugfix] Fix missing is_layer_skipped check for FusedMoE in AWQConfig (#32935)

joninco
authored and
cwazai
committed
[CI] fix version comparsion and exclusion patterns in upload-release-wheels.sh (#32971)

Harry-Chen
authored and
cwazai
committed
[fix] add VLLM_OBJECT_STORAGE_SHM_BUFFER_NAME to compile factors (#32912)

dolpm
authored and
cwazai
committed
[Performance] Split FlashAttn attention and cache update (#25954)

authored and
cwazai
committed
[Core][Bugfix] allow graceful worker termination (#32965)

joerunde
authored and
cwazai
committed
[ROCm][ViT] Enable Flash Attention Triton backend on RDNA3/RDNA4 (#32944)

monajafi-amd
authored and
cwazai
committed
Auth_token added in documentation as it is required (#32988)

authored and
cwazai
committed
[Dev UX] Add auto-detection for VLLM_PRECOMPILED_WHEEL_VARIANT during install (#32948)

authored and
cwazai
committed
[Perf] Cache xpu_get_mem_info() result to avoid duplicate calls (#32983)

sjhddh
authored and
cwazai
committed
[Tests] Clarify pytest skip reasons with actionable context (#32981)

sjhddh
authored and
cwazai
committed
[Tests] Standardize RNG seed utility across test files (#32982)

sjhddh
authored and
cwazai
committed
[docs] Update governance process links (#32995)

authored and
cwazai
committed
[Models] Add `SharedFusedMoE` support to Qwen3MoE (#32082)

Isotr0py
authored and
cwazai
committed
[Doc] Ignore typo check on doc (#32999)

ywang96
authored and
cwazai
committed
feat(benchmark): add encoder forward pass benchmarking to mm-processor (#31655)

authored and
cwazai
committed
[UX] Deduplicate sampling parameter startup logs (#32953)

DarkLight1337
authored and
cwazai
committed
[Perf] Cache exc.errors() result in validation exception handler (#32984)

sjhddh
authored and
cwazai
committed
[Bugfix] Fix E2E latency calculation and add warmup support in mm_processor benchmark (#32646)

authored and
cwazai
committed
[Models]: Make Multimodal config implicit in ViT implementation (#31972)

Isotr0py
authored and
cwazai
committed
feat: Complete LoRA support for MiniMaxM2 Fixes #32736 (#32763)

authored and
cwazai
committed
[EncoderCacheManager] Remove unnecessary copy (#32800)

lgeiger
authored and
cwazai
committed
[Bugfix]: resolve torch.compile cache conflict between mm_encoder_tp_modes (#32842)

authored and
cwazai
committed
Update CPU doc according to feedback (#32963)

authored and
cwazai
committed
[MLA] Fuse cat and qaunt for fp8 kv-cache (#32950)

LucasWilkinson
authored and
cwazai
committed
[Tests] Replace flaky sleep with polling in test_background_cancel (#32986)

sjhddh
authored and
cwazai
committed
[CPU Backend][BugFix] Fix failing Darwin pipelines (#33002)

fadara01
authored and
cwazai
committed
[CPU] Improve CPU Docker build (#30953)

authored and
cwazai
committed
[Feature] add session based streaming input support to v1 (#28973)

authored and
cwazai
committed
[DOC] [ROCm] Update doc for v0.14.1 (#32998)

tjtanaa
authored and
cwazai
committed
[Perf][Kernel] Optimize FP4 quantization kernels (SM100F) (#32520)

LopezCastroRoberto
authored and
cwazai
committed
[Docs] Fix Apple silicon include path in CPU installation docs (#32977)

sjhddh
authored and
cwazai
committed
[Bugfix] fix encoder cache hang in Qwen3VL (#32684)

authored and
cwazai
committed
[BugFix] Add env variable to control PDL in LoRA (#32836)

jeejeelee
authored and
cwazai
committed
Merge branch 'main' into pr1/moe-lora-kernel-opt
cwazai
authored

Commits on Jan 26, 2026

Merge branch 'main' into pr1/moe-lora-kernel-opt
jeejeelee
authored