fix(mooncake): address review comments on HBM leak fix#1

Merged

machov merged 461 commits intofix/vllm-35943-mooncake-hbm-leakfrom

copilot/fix-mooncake-hbm-leak-temp

Mar 10, 2026

This pull request is big! We're only showing the most recent 250 commits

Commits on Mar 10, 2026

[CI][AMD][BugFix] Add torch.cuda.set_device to test_punica_ops so punica kernels execute on same device as tensor (#34985)

rasmith
authored and
Copilot
committed
fix(mxfp4): Disable monolithic path for TRITON backend with EP (#34270)

authored and
Copilot
committed
[ROCm][CI] Extending attention backend coverage for Eagle spec decode tests (#35265)

AndreasKaratzas
authored and
Copilot
committed
[ROCm][CI] Amending deletion of AMD mirror (#35322)

AndreasKaratzas
authored and
Copilot
committed
[offloader] v2: Hide weight onloading latency via prefetching (#29941)

authored and
Copilot
committed
[UX] Add `--performance-mode {balanced,interactivity,throughput}` (#34936)

mgoin
authored and
Copilot
committed
[MoE Refactor] MXFP4 Cutlass Experts to MK (#34542)

zyongye
authored and
Copilot
committed
[UX] Add `--moe-backend` arg for explicit kernel selection (#33807)

authored and
Copilot
committed
[Bugfix] Fix AttributeError in SMControlContextManager (#35338)

LucasWilkinson
authored and
Copilot
committed
[Bugfix] Fix CUDA compatibility path setting for both datacenter and consumer NVIDIA GPUs (#33992)

authored and
Copilot
committed
[BugFix] Fix fp4 quant kernel on CUDA 12.8 (#35210)

LopezCastroRoberto
authored and
Copilot
committed
openpangu-vl support video input (#34134)

authored and
Copilot
committed
[Kernel] Refactor FlashInfer allreduce for mnnvl backend (#34109)

authored and
Copilot
committed
[Model Runner V2] Add coding style guide (#35325)

WoosukKwon
authored and
Copilot
committed
[CPU][Feat] Enable KleidiAI INT8_W4A8 for all input dtypes (#34890)

authored and
Copilot
committed
[torch.compile] Sequence Parallelism threshold compile ranges (#28672)

authored and
Copilot
committed
[BugFix] anthropic/serving_messages: fix tool call arguments streaming (#34887)

authored and
Copilot
committed
[Misc][Harmony] Move Responses API only harmony utils to responses/harmony.py (#35339)

sfeng33
authored and
Copilot
committed
[Benchmark] Simplify SLA scan (#35306)

DarkLight1337
authored and
Copilot
committed
[BugFix][XPU] Fix speculative decoding on Intel XPU due to bug with `IGC_ForceOCLSIMDWidth=16` (#35298)

authored and
Copilot
committed
[XPU][8/N] Fix kernel bugs in XPU LoRA and MOE LORA (#34115)

authored and
Copilot
committed
[ROCm] Add extra step in config initialization to populate custom ops before compilation config init (#34848)

gshtras
authored and
Copilot
committed
[Bugfix][Hardware][AMD] Gate FP4 ops on gfx950 to prevent MI300X crash (#35250)

c0de128
authored and
Copilot
committed
[Test] Add tests for n parameter in chat completions API (#35283)

KrxGu
authored and
Copilot
committed
[Model] Ring 2.5 (#35102)

ZJY0516
authored and
Copilot
committed
[Benchmarks] Plot benchmark timeline and requests statistics (#35220)

authored and
Copilot
committed
[Hardware][Powerpc]Enable prefix caching and chunked prefill for ppc64le (#35081)

authored and
Copilot
committed
[Bugfix] [Qwen3.5]Fix Qwen3.5 FP8 quantization: tuple shard_id weight loading (#35289)

authored and
Copilot
committed
[XPU] use fixed UMD version in dockerfile.xpu (#35392)

jikunshang
authored and
Copilot
committed
Remove `bc-lint` (#35274)

hmellor
authored and
Copilot
committed
[Bugfix] Fix Qwen2.5-Omni and Qwen3-Omni mixed-modality embed regression (#35368)

linyueqian
authored and
Copilot
committed
[Bugfix] fix device_name for routing replay (#34336)

Li-Yongwen
authored and
Copilot
committed
[Bugfix] Fix uint32 overflow in Mamba selective scan state pointer arithmetic (#35275)

Josephasafg
authored and
Copilot
committed
[Misc] Standardize handling of `mm_processor_kwargs.size` (#35284)

DarkLight1337
authored and
Copilot
committed
[Bug] Fix missing <think> tag after tool call in MiniMax 2.1 (#35352)

stingoChen
authored and
Copilot
committed
[Model] Add nvidia/llama-nemotron-embed-vl-1b-v2 multimodal embedding model (#35297)

jzakrzew
authored and
Copilot
committed
[Refactor] Remove dead or duplicate func utils or variables (#35318)

yewentao256
authored and
Copilot
committed
[ROCm] Add dynamic mxfp4 quantization for DeepSeek V2 projection layers (#34157)

authored and
Copilot
committed
[ROCm] Update the torch version in rocm_build.txt to use the official 2.10 release (#34387)

SageMoore
authored and
Copilot
committed
[BugFix][kv_offload]: Fix kernel block size detection (#35125)

orozery
authored and
Copilot
committed
Add GlmOcrConfig for GLM-OCR model type recognition (#34982)

hujia177
authored and
Copilot
committed
[Perf] Optimize maxsim scores computation for pooling models, 13.9% E2E throughput improvement (#35330)

yewentao256
authored and
Copilot
committed
[Core] Support `min_tokens` with speculative decoding (#32642)

authored and
Copilot
committed
[Refactor] Remove dead code for attention benchmark script (#35418)

yewentao256
authored and
Copilot
committed
[BugFix] Align fused MoE-LoRA kernel config with actual weight shapes (#34396)

RunkaiTao
authored and
Copilot
committed
[Bugfix] Remove erroneous lower bound on LoRA vocab size constraint (#35354)

LucasWilkinson
authored and
Copilot
committed
[Model Runner V2] Fix error-handling (#35063)

njhill
authored and
Copilot
committed
[Model Runner V2] Add model states [1/N] (#35350)

WoosukKwon
authored and
Copilot
committed
[Model Runner V2] Prepare attn metadata in ModelState [2/N] (#35383)

WoosukKwon
authored and
Copilot
committed
fix(reasoning): Qwen3ReasoningParser returns truncated output as reasoning (#35230)

stakeswky
authored and
Copilot
committed
[Performance] Extract KV cache update op from flashinfer forward (#35422)

ElizaWszola
authored and
Copilot
committed
Nemotron: use per-layer config in NemotronHMLPDecoderLayer for heterogeneous models (#35396)

danielafrimi
authored and
Copilot
committed
add mixed precision support for modelopt (#35047)

sychen52
authored and
Copilot
committed
[Bugfix] Fix MessageQueue connect_ip for cross-node data parallelism (#35429)

authored and
Copilot
committed
[WideEP] Remove pplx all2all backend (#33724)

authored and
Copilot
committed
[Kernel][perf] optimize NCCL symm_mem vs custom_AR selection thresholds (#33839)

authored and
Copilot
committed
[ROCm][Quantization] GPT OSS Upstream MoE wmxfp4_afp8 with static scales (#30357)

authored and
Copilot
committed
[Bugfix] Fix KV Scale loading for MLA Models (#35430)

pavanimajety
authored and
Copilot
committed
[Update] Use FlashInfer fast_decode_plan directly instead of replication (#34687)

authored and
Copilot
committed
[Performance] Cublas Bf16 Gate with Fp32 Output (#35121)

roikoren755
authored and
Copilot
committed
[CI] Actually run tests/kernels/quantization/test_block_fp8.py in CI (#34274)

mgoin
authored and
Copilot
committed
[Bugfix] Emit reasoning_part events in simple streaming path for Resp… (#35184)

daniel-salib
authored and
Copilot
committed
[compile] Invalidate cache for cpu flags (#35119)

angelayi
authored and
Copilot
committed
[Core]Extract is_last_rank in Ray for tpu to override (#33012)

Chenyaaang
authored and
Copilot
committed
[Misc] Move `GPUModelRunner.prepare_kernel_block_sizes` to utils (#35400)

NickLucche
authored and
Copilot
committed
[Bugfix] Fix Qwen3NextForCausalLM packed_modules_mapping (#35413)

jeejeelee
authored and
Copilot
committed
use 'max_active_experts' for moe lora input size (#33197)

gnovack
authored and
Copilot
committed
[Bug] Fix outdated links in source code (#35314)

yewentao256
authored and
Copilot
committed
[BugFix] Repo utils debug print patch (#35434)

pi314ever
authored and
Copilot
committed
[Bugfix] disable allreduce_rms_fusion by default when pp size > 1 (#35424)

ZJY0516
authored and
Copilot
committed
[Bug] correct out dtype of rms_norm_gated native path (#35369)

authored and
Copilot
committed
[Model Performance] Add Qwen3MoE tuned MoE configs for H200 (#35457)

authored and
Copilot
committed
[Bugfix] Use 'sum' reduction instead of 'avg' in Async TP reduce-scatter (#33088)

authored and
Copilot
committed
[Bugfix] Replace assert with ValueError for response_format validation in completions endpoint (#35456)

umut-polat
authored and
Copilot
committed
Flashinfer cuDNN backend for Qwen3 VL ViT attention (#34580)

authored and
Copilot
committed
[Bugfix] Add missing activation attr to RMSNormGated (#35423)

authored and
Copilot
committed
[compile] Cleanup: Remove unnecessary +rms_norm forcing for sequence parallelism (#35410)

jasonlizhengjian
authored and
Copilot
committed
Revert "Add GlmOcrConfig for GLM-OCR model type recognition" (#35512)

hmellor
authored and
Copilot
committed
[Bugfix] Fix DCP + FA3 crash due to missing num_splits in _forward_with_dcp (#35082)

haosdent
authored and
Copilot
committed
[Bugfix] Fix check_interleaved_audio_video false positive for batched non-interleaved requests (#35487)

authored and
Copilot
committed
[Bugfix] Handle case when kimi ends reasoning with a tool call (#33646)

authored and
Copilot
committed
Add @BoyuanFeng to CODEOWNERS (#35317)

BoyuanFeng
authored and
Copilot
committed
[Core] Fix `gpu_worker.py` pre-commit errors (#35312)

njhill
authored and
Copilot
committed
[Kernel] [Helion] [7/N] Use HOP to represent Helion Kernel call to enable fx tracing and pattern matching (#34390)

gmagogsfm
authored and
Copilot
committed
[Model] Add huggingface skt/A.X-K1 model (#32407)

authored and
Copilot
committed
[Misc] Fill in some v1 CODEOWNERS gaps (#35524)

njhill
authored and
Copilot
committed
[perf] Use pinned memory for async H2D transfer in do_mamba_copy_block (#35480)

hl475
authored and
Copilot
committed
[Doc] Fix link to Llama chat template for usability (#35525)

authored and
Copilot
committed
Support parakeet as audio encoder for nemotron-nano-vl (#35100)

authored and
Copilot
committed
[BugFix] Fix 3D rope in transformers backend (#35097)

authored and
Copilot
committed
[Model Runner V2] Warmup kernels (#35172)

njhill
authored and
Copilot
committed
[compile] Fix caching error over pytree slice node. (#35308)

zhxchen17
authored and
Copilot
committed
[Transformers backend] Ignore MTP weights when num_nextn_predict_layers=0 (#34888)

authored and
Copilot
committed
[Bugfix] Add monkeypatch to prevent race condition from writing (#35420)

Lucaskabela
authored and
Copilot
committed
[DP] Only use DP padding when cudagraphs are actually used (#34102)

LucasWilkinson
authored and
Copilot
committed
[Bugfix][Model] Fix gpt-oss batch invariance (#35404)

jzakrzew
authored and
Copilot
committed
[Feat][RL][2/2] Native Weight Syncing API: IPC (#34171)

hao-aaron
authored and
Copilot
committed
[ROCm] Enabling encoder and encoder-decoder on ROCm and AITER unified backends (#35334)

gshtras
authored and
Copilot
committed
[ROCm]: fix aiter rope functionalization (#35533)

Rohan138
authored and
Copilot
committed
[Refactor][Kernel] Add global helper to deduplicate vectorized memory ops (#35105)

LopezCastroRoberto
authored and
Copilot
committed
[misc] cleanup one level of error stack when nixl fails to initialize (#35517)

youkaichao
authored and
Copilot
committed
[Misc] Clean up ResponsesRequest model validators (#35531)

umut-polat
authored and
Copilot
committed
[Model Runner V2] Support pooling models (#35120)

WoosukKwon
authored and
Copilot
committed
[Model Runner V2] Move MM encoder to Model States [3/N] (#35564)

WoosukKwon
authored and
Copilot
committed
[ROCm][Quantization] Add Composable Kernel (CK) backend support for M… (#34301)

authored and
Copilot
committed
[ROCm] Add `stablelm` Head Size 80 To Supported Head Sizes For ROCM_ATTN (#35527)

micah-wil
authored and
Copilot
committed
[Bugfix] Fixes for SLA finder (#35537)

DarkLight1337
authored and
Copilot
committed
[MTP] Validate that MTP weights are actually loaded (#35548)

MatthewBonanni
authored and
Copilot
committed
[CI/Build] CPU release supports both of AVX2 and AVX512 (#35466)

authored and
Copilot
committed
[1/N] Elastic EP Milestone 2 (#34861)

authored and
Copilot
committed
[Bugfix] Propagate compilation_time from workers to main process for TP>1 (#35503)

huydhn
authored and
Copilot
committed
[Bugfix] Move chat completion response_format validation to Pydantic model_validator (#35510)

umut-polat
authored and
Copilot
committed
[EPLB] Enforce sync eplb for NCCL-based all2all backend (#35212)

ilmarkov
authored and
Copilot
committed
[ROCm][CI] Adding infiniband mappings for moriio tests (#35170)

AndreasKaratzas
authored and
Copilot
committed
[ROCm] Derive device capability from GCN arch string without CUDA init (#35069)

AndreasKaratzas
authored and
Copilot
committed
[ROCm][CI] Expose tests to AMD production CI and fix amdsmi heap corruption (#35071)

AndreasKaratzas
authored and
Copilot
committed
[CI] add trainer_send_weights for MockWeightTransferEngine (#35589)

chaunceyjiang
authored and
Copilot
committed
[Misc] Change logging level from info to debug for tool parser import (#35575)

chaunceyjiang
authored and
Copilot
committed
[Benchmark] Rename SLA Finder to Workload Explorer (#35586)

DarkLight1337
authored and
Copilot
committed
[Benchmark] Improve UX of sweep scripts (#35600)

DarkLight1337
authored and
Copilot
committed
[ROCm][CI] Parametrize vision score tests across attention backends with per-backend tolerances (#35571)

AndreasKaratzas
authored and
Copilot
committed
Add padding support to wvSplitK solution for skinny GEMMs (#33762)

amd-hhashemi
authored and
Copilot
committed
[Feature]Supports Anthropic Thinking Block (#33671)

authored and
Copilot
committed
add io_process_plugin for sparse embedding (#34214)

authored and
Copilot
committed
[Feat] Add CUDA torch fallbacks for fp8_mqa_logits/fp8_paged_mqa_logits_torch function (#35271)

chaunceyjiang
authored and
Copilot
committed
custom dataset img support base64 (#35280)

flutist
authored and
Copilot
committed
Fix Qwen3_5MTP packed_modules_mapping for gate_up_proj (#35581)

cwazai
authored and
Copilot
committed
[Fix] Avoid sending image input to other PP ranks (#35405)

authored and
Copilot
committed
[Benchmark] Avoid unnecessary video download in MMVU (#35618)

DarkLight1337
authored and
Copilot
committed
[Deprecation] Deprecate code in 0.17 as scheduled (#35441)

authored and
Copilot
committed
[Chore] Cleanup BNB utilization dead code (#35620)

Isotr0py
authored and
Copilot
committed
[Bugfix] Fix Anthropic API base64 image handling in Messages endpoint (#35557)

voipmonitor
authored and
Copilot
committed
[Model Runner V2] Add ModelStateInterface [4/N] (#35621)

WoosukKwon
authored and
Copilot
committed
Add TMA support to fused_moe_lora kernel (#32195)

authored and
Copilot
committed
[Bugfix][Model] Fix Qwen3.5/Qwen3Next ignoring --dtype flag on older GPUs (#35617)

lailoo
authored and
Copilot
committed
Fix typo: implictly -> implicitly in isaac.py docstring (#35646)

lin-shh
authored and
Copilot
committed
[AMD][CI] Support Triton attention with ExampleConnector (#34931)

rjrock
authored and
Copilot
committed
[Model Runner V2] Minor refactoring for EncoderRunner (#35628)

WoosukKwon
authored and
Copilot
committed
[Mamba1] - Kernel Level Chunk Alignment for Prefix Caching (#34798)

Josephasafg
authored and
Copilot
committed
[MISC] Fixing a null reference by removing parallel_utils from mypy EXCLUDE (#35630)

taneem-ibrahim
authored and
Copilot
committed
fix(mxfp4): return is_monolithic=False when LoRA is enabled for Triton backend (#35382)

yoonsnowdev
authored and
Copilot
committed
[Model Runner V2] Use block table apis for capture inputs (#35671)

WoosukKwon
authored and
Copilot
committed
[Bugfix] Fix dtype mismatch in RMSNormGated.forward_native() during torch.compile (#35256)

haosdent
authored and
Copilot
committed
[torch.compile] Undo the fast_moe_cold_start hack in torch>=2.11 (#35475)

zou3519
authored and
Copilot
committed
Revert "[Bugfix] Disable TRTLLM attention with KV transfer enabled (#33192)" (#34832)

ZhanqiuHu
authored and
Copilot
committed
[Attention] FA4 integration (#32974)

authored and
Copilot
committed
Fix deprecated v1 config tests (#35327)

jcaip
authored and
Copilot
committed
[XPU] fix mxfp4 activation type (#35691)

jikunshang
authored and
Copilot
committed
[ROCm] add amd-quark package in requirements for rocm to use quantized models (#35658)

authored and
Copilot
committed
[ROCm][CI] Disable skinny GEMMs in language model standard tests to fix non-determinism (#35152)

AndreasKaratzas
authored and
Copilot
committed
[Kernel] Integrate SM100 MXFP8 blockscaled grouped MM and quant kernels (#34448)

authored and
Copilot
committed
[Rocm][CI] Fix LM Eval Large Models (H100) test group (#34750)

charlifu
authored and
Copilot
committed
[CI] Defining extended V1 e2e + engine tests (#35580)

AndreasKaratzas
authored and
Copilot
committed
[Misc] Bound NIXL upper bound version (#35495)

NickLucche
authored and
Copilot
committed
[CPU][Distributed] Fix Enable _CPUSHMDistributed only when TP/PP ranks share the same SHM group name (#34169)

charlesashby
authored and
Copilot
committed
[Misc] Cleanup useless `current_platform` import (#35715)

wangxiyuan
authored and
Copilot
committed
[Feat] Supports Anthropic Messages count_tokens API (#35588)

chaunceyjiang
authored and
Copilot
committed
Fix unresolved-import errors when using Astral's ty by removing src.root (#35681)

tlrmchlsmth
authored and
Copilot
committed
[MyPy][BugFix] Check profiler is assigned before calling start() on it (#35505)

authored and
Copilot
committed
[Docs] Add breadcrumbs for better UX (#35749)

hmellor
authored and
Copilot
committed
[Fix Bug]`num_active_loras` always equals to zero (#34119)

authored and
Copilot
committed
[Performance] Extract kv update ops from MLA attention backends (#34627)

authored and
Copilot
committed
[CI] Fix mypy for vllm/device allocator (#35518)

authored and
Copilot
committed
[Core] Move test utility to test file (#35672)

wjabbour
authored and
Copilot
committed
[Doc] Improve UX of `--enable-log-requests` (#35723)

DarkLight1337
authored and
Copilot
committed
[CI][HPU] Pin vllm commit compatible with vllm-gaudi - HPU tests (#35307)

PatrykWo
authored and
Copilot
committed
[CI/Build] Enable Qwen3.5 tests on CI (#35763)

Isotr0py
authored and
Copilot
committed
[BugFix][Model]Fix the garbled code in Ernie4.5-VL caused by fast_moe_cold_start (#35587)

CSWYF3634076
authored and
Copilot
committed
[torch.compile] Improve cold and warm start compile tests (#35709)

zou3519
authored and
Copilot
committed
[Spec Decode] Add hidden states extraction system (#33736)

fynnsu
authored and
Copilot
committed
[KVConnector] Auto-downgrade to PIECEWISE cudagraph mode for layerwise async ops (#31057)

yashwantbezawada
authored and
Copilot
committed
[ci] Add Ray compatibility check informational CI job (#34672)

jeffreywang-anyscale
authored and
Copilot
committed
[BUG] Fix rlhf_async example (#35788)

hao-aaron
authored and
Copilot
committed
[Bugfix] Fix missing sequence_lengths in qwen3_omni_moe_thinker (#35741)

yeqcharlotte
authored and
Copilot
committed
clean unused cudagraph_batch_sizes (#35552)

BoyuanFeng
authored and
Copilot
committed
[MoE][Perf] Wrap DSV3 QKVAProj GEMM in custom op for torch.compile (#35751)

authored and
Copilot
committed
[Bugfix] Fix MM processor test for Qwen3.5 (#35797)

ywang96
authored and
Copilot
committed
[All Reduce] Change default backend of Flashinfer All Reduce to trtllm (#35793)

hjjq
authored and
Copilot
committed
[ROCm][CI] Fix backslash-continuation in pytest marker re-quoting and treat exit code 5 as success (#35798)

AndreasKaratzas
authored and
Copilot
committed
[Model] Add support for nvidia/llama-nemotron-rerank-vl-1b-v2 (#35735)

jzakrzew
authored and
Copilot
committed
[XPU][NIXL] Add GPUDirect RDMA support for XPU (#35270)

authored and
Copilot
committed
[Model Runner V2][Perf] align dummy_run tokens to uniform decode for dp cudagraph (#35376)

izhuhaoran
authored and
Copilot
committed
[Tool Parser] Fix Qwen3Coder streaming parameter loss with speculative decode (#35615)

authored and
Copilot
committed
[CI] Temporarily Disable Nightly Failures (#35770)

authored and
Copilot
committed
[ModelRunnerV2] Rename sampler functions and variables for clarity (#35459)

andylolu2
authored and
Copilot
committed
[Docs][Model Runner V2] Add Design Docs (#35819)

WoosukKwon
authored and
Copilot
committed
[V0 deprecation] Remove Swin model (#35821)

Isotr0py
authored and
Copilot
committed
[Model Runner V2] Use ModelState.prepare_attn() for cuda graph capture [5/N] (#35774)

WoosukKwon
authored and
Copilot
committed
[Core] Add optional flags to check for repetitive token patterns in engine output (#35451)

aykoppol
authored and
Copilot
committed
[CI/Build] Automatically patch video metadata for multimodal processor test (#35822)

Isotr0py
authored and
Copilot
committed
[Refactor] Fix maxsim cuda platform and add cli to control it (#35427)

authored and
Copilot
committed
[ROCm][CI] Fix Assertion Logic For `test_gpt_oss` (#35806)

micah-wil
authored and
Copilot
committed
[CI/Build] Trigger processor tests on registry update (#35824)

DarkLight1337
authored and
Copilot
committed
[BugFix] Fix cmake based incremental install (wrong vllm install dir) (#35773)

authored and
Copilot
committed
[MISC] Removed unused function find_all_indices() from tool_parsers/utils.py (#35683)

taneem-ibrahim
authored and
Copilot
committed
[Misc] Fix typos in comments: explict→explicit, paramaters→parameters (#35648)

lin-shh
authored and
Copilot
committed
Fix TYPE_CHECKING stub defaults in envs.py to match actual runtime defaults (#35645)

lin-shh
authored and
Copilot
committed
[ROCm] [Release] Change the package from `aiter` to `amd-aiter` (#35198)

tjtanaa
authored and
Copilot
committed
add regression test (#35834)

hallerite
authored and
Copilot
committed
[CI/Build][Intel] Add new performance benchmarks for Intel Gaudi 3 (#31025)

authored and
Copilot
committed
[Perf] [Hybrid] Copy num_accepted_tokens in non-blocking way when not using prefix caching (#35442)

tdoublep
authored and
Copilot
committed
[CI] And PPL test for Qwen3.5. (#35853)

authored and
Copilot
committed
[Bugfix] Avoid src/dst as None in irecv/isend_tensor_dict (#35754)

bigPYJ1151
authored and
Copilot
committed
[Frontend][1/n] Improve pooling entrypoints | classify. (#35604)

authored and
Copilot
committed
[ROCm] [CI] Add new fusion test cases that are relevant to vLLM IR Ops (#34307)

authored and
Copilot
committed
[BugFix] Add support for MTP num_speculative_tokens > 1 with sparse MLA (#34552)

authored and
Copilot
committed
TRTLLM gen-full attn Test Coverage (#34986)

authored and
Copilot
committed
fix: Ensure invalid audio files return 400 error (#34715)

authored and
Copilot
committed
[CI] Bump `num_speculative_tokens` to 3 in nightly DeepSeek tests (#35882)

MatthewBonanni
authored and
Copilot
committed
[CI] Temporarily Disable Llama4 MoE Refactor Test (#35870)

robertgshaw2-redhat
authored and
Copilot
committed
[MoE Refactor] Create MK for TRTLLM Kernels (#32564)

authored and
Copilot
committed
[ROCm][Bugfix]: Disable AITER Triton ROPE by default (#35601)

Rohan138
authored and
Copilot
committed
[ROCm][CI] Fix TP size issue for `test_gpt_oss` (#35887)

micah-wil
authored and
Copilot
committed
[Bugfix] Fix misnamed parameter in compressed_tensors_moe.py (#35813)

authored and
Copilot
committed
[Model Runner V2] Fix inputs_embeds=None bug for MM models (#35917)

WoosukKwon
authored and
Copilot
committed
[CI/Build] Allow mounting AWS credentials for sccache S3 auth (#35912)

amrmahdi
authored and
Copilot
committed
[Model Runner V2] support dp & ep for spec decoding (#35294)

authored and
Copilot
committed
[Core] Move save_tensorized_model logic to Worker (#35825)

njhill
authored and
Copilot
committed
[Bugfix] Fix coord_socket assertion in DPEngineCoreProc for offline DP mode (#35916)

jaewonlee-fb
authored and
Copilot
committed
[ROCm][CI] Support async weight transfer example with platform-aware determinism (#35710)

AndreasKaratzas
authored and
Copilot
committed
Enable bnb for multiple indices weight (#35838)

authored and
Copilot
committed
[Bugfix] Fix EVS implementation for Qwen3 VL (#33607)

2ez4bz
authored and
Copilot
committed
[PluggableLayer][MM] Add PluggableLayer for RelPosAttention (#33753)

shen-shanshan
authored and
Copilot
committed
[model] support FireRedASR2 (#35727)

authored and
Copilot
committed
[Chore] Remove debug code in model implementation (#35883)

Isotr0py
authored and
Copilot
committed
[Refactor] Clean up processor kwargs extraction (#35872)

DarkLight1337
authored and
Copilot
committed
[Bugfix] Guard mm_token_type_ids kwarg in get_mrope_input_positions (#35711)

AndreasKaratzas
authored and
Copilot
committed
[Rocm][CI] Fix ROCm LM Eval Large Models (8 Card) (#35913)

charlifu
authored and
Copilot
committed
[BugFix] Support tool_choice=none in the Anthropic API (#35835)

authored and
Copilot
committed
[Bugfix] Improve engine ready timeout error message (#35616)

lailoo
authored and
Copilot
committed
[cohere][fix][spec-decode]: fix crash when allowed_token_ids is set without penalties (#35654)

kkt-cohere
authored and
Copilot
committed
Support Audio Extraction from MP4 Video for Nemotron Nano VL (#35539)

authored and
Copilot
committed
[Core] Remove busy loop from idle buffer readers (#28053)

authored and
Copilot
committed
[Bugfix] Add missing dynamic_arg_dims for Qwen3-ASR torch.compile (#35869)

authored and
Copilot
committed
[ROCm][Bugfix] Fall back from CK MXFP4 MoE when GEMM dimensions are unsupported (#35893)

ChuanLi1101
authored and
Copilot
committed
[Hardware] Replace `torch.cuda.empty_cache` with `torch.accelerator.empty_cache` (#30681)

authored and
Copilot
committed
[XPU] bump vllm-xpu-kernels to v0.1.3 (#35984)

jikunshang
authored and
Copilot
committed
[Bugfix] Cap FULL decode cudagraph sizes for Mamba/hybrid models (#34094) (#34571)

authored and
Copilot
committed
[Feature] Add basic metrics for /realtime endpoint (#35500)

authored and
Copilot
committed
[MISC] fixed tool_parser mypy errors (#35640)

authored and
Copilot
committed
[Bugfix][CPUOffloadingManager] Prevent eviction of already-stored blocks in LRU/ARC `prepare_store()` (#35846)

ronensc
authored and
Copilot
committed
[Bugfix][Model] Fix FP8 k_scale/v_scale not loaded for Qwen3-MoE (#35656)

oneraghavan
authored and
Copilot
committed
[BugFix] Fix implicit and incorrect assumption on ECConnector is_producer (#34783)

furionw
authored and
Copilot
committed
[Bugfix] Make `kaldi_native_fbank` optional (#35996)

DarkLight1337
authored and
Copilot
committed
fix(mooncake): resolve HBM leak from stuck WAITING_FOR_REMOTE_KVS requests

machov
authored and
Copilot
committed
fix: address CI feedback - line length and missing metadata attribute

machov
authored and
Copilot
committed
fix(mooncake): resolve race conditions in KV transfer handling

machov
authored and
Copilot
committed
fix(mooncake): address review comments from PR #36014

Copilot
and
machov
committed
fix(mooncake): address review comments from PR #36014

Copilot
and
machov
committed
Merge remote-tracking branch 'origin/copilot/fix-mooncake-hbm-leak-temp' into copilot/fix-mooncake-hbm-leak-temp
Copilot
committed