[PCP] Support prefill context parallelism with data parallelism by LucasWilkinson · Pull Request #57075 · vllm-project/vllm

LucasWilkinson · 2026-09-15T21:03:43Z

Enable prefill context parallelism (PCP) with data parallelism (DP), including expert-parallel MoE and MTP decoding.

Reuse existing communication groups, add sequence-parallel GLM MoE support, and fix DP padding and idle-rank handling in PCP draft execution. No new process group.

Continues #54131 with the original authorship preserved; supersedes the earlier approach in #49246.

Validation:

CUDA_VISIBLE_DEVICES='' .venv/bin/python -m pytest -q tests/test_pcp_dp.py: 16 passed. Broader focused suite: 170 passed, 10 GPU-only skips.
git diff --name-only f2aad6aa70 -z | xargs -0 .venv/bin/pre-commit run --files: passed.
GLM-4.7-Flash, MTP3, full decode CUDA graphs: GSM8K 52/64 with EP (TP2×PCP2×DP2), 54/64 without EP (TP1×PCP2×DP2). Both passed all nine uneven-load/idle-rank stress requests. An initial non-EP run had one unparseable answer; it did not reproduce.

ROCm and performance parity remain untested. These GSM8K runs are smoke evaluations.

AI assistance: Claude and Codex.

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>

- main's sync_cudagraph_and_dp_padding all-reduces six rows (allow_ubatching and num_reqs were added after this series branched), so the fake all_reduce in the V2 batch-coordination test has to match. - Reformat the elastic-EP standby block the rebase left misformatted. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

vllm-project#54523 moved the ParallelConfig check into CudaPlatform/RocmPlatform check_and_update_config after this series branched; the series removes the ParallelConfig copy, so remove the platform copies too. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

Under DP the batch is padded to the DP-agreed token count, so a decode-only batch can carry more rows than attn_metadata.num_decode_tokens. The PCP cache gather then treated the padding rows as partitioned prefill tokens and failed on the replicated slot mapping (shape '[8, 6]' is invalid for input of size 6). When num_decode_tokens == num_actual_tokens, write only the real decode rows. Prefill batches are left on their padded rows: that padding is uniform across PCP ranks and the all-gather relies on it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

With expert parallelism, each PCP rank holds its own expert shard, so MoE dispatch and the token-count coordination feeding it run over get_moe_non_sp_group() rather than the DP group. The series spelled that condition out at five call sites; give it a name on ParallelConfig instead. No behavior change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

vllm-project#51700 (FULL CUDA graphs for DBO) landed after this series branched, and its microbatching branch builds a DPSyncState without moe_non_sp_token_counts, which the series makes a required field. Any DP run that microbatched raised TypeError, with or without PCP; main's own microbatching tests catch it. That branch also decided microbatching from this PCP coordinate's DP view. Decide from the counts every rank in the sync group reported, so all PCP coordinates reach the same answer. DPSyncState.padded_to() collects the "pad every rank to the agreed count" pattern that the new field would otherwise duplicate at each site. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

The draft forward contexts were built from the bare num_tokens_across_dp tensor, so under DP x PCP with expert parallelism the MoE non-SP token counts never reached DPMetadata.make and the MTP draft's profiling run asserted. Pass the DPSyncState down instead, in the autoregressive and multi-module MTP speculators and in the draft CUDA graph capture that drives both. uniform_dp_token_counts() builds the counts for a batch every rank runs identically; model capture and draft capture now share it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

Derive PCP dispatch sizes from the V2 runner's padded DP counts instead of threading a second count tensor through target and draft forwards. Reuse existing DP or PCP communicators when possible and remove unsupported legacy-runner and DBO plumbing. Keep replicated draft cache writes local with DP padding. Pad the draft prefill mask to the graph request count and classify zero-length padding as decode requests. Validation: 175 focused tests passed with GPU tests enabled; pre-commit and diff checks passed. GLM-4.7-Flash MTP3 GSM8K smoke evaluations scored 53/64 with DP2/PCP2/TP2 full decode graphs and 54/64 with DP2/PCP2 DeepEP high throughput. Both configurations passed uneven-load and idle-rank requests with zero invalid responses or GPU sync-check exceptions. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

Keep a single DP token-count tensor and derive MoE dispatch sizes from PCP, sequence parallelism, and EP configuration. With EP, repeat DP counts across PCP ranks; without EP, account for the PCP tokens already gathered before DP dispatch. Expose the combined group through get_dp_group(include_pcp=True), reuse existing groups where possible, and remove the MoE non-SP abstraction. Clear stale PCP draft batches before preparing dummy forwards so an idle DP rank cannot reuse a previous graph batch's shape. Validation: 178 focused tests passed on GPU; all pre-commit checks passed. GLM-4.7-Flash MTP3 with DP2/PCP2 and full decode graphs scored 51/64 GSM8K with EP and 53/64 without EP. Both passed all uneven-load and idle-rank requests, with zero invalid responses or inference sync-check exceptions. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

Remove the DP-PCP process group and its lifecycle plumbing. Enable sequence-parallel GLM MoE inputs for PCP configurations so TP ranks can share the existing EP dispatch/combine stage, with replicated shared expert weights. Validation: 170 focused tests passed, 10 GPU-only tests skipped, and pre-commit passed. Eight-GPU GLM-4.7-Flash TP2/PCP2/DP2 with EP, MTP3 and full decode graphs scored 52/64 GSM8K with zero invalid responses and passed all nine uneven-load stress requests. Performance parity has not been benchmarked. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

Pass PCP size and EP mode from the MoE config instead of storing duplicate fields in DPMetadata. Expand PCP counts before the original SP calculation and name the worker rank stride to include PCP. Validation: 170 focused tests passed, 10 GPU-only skips; pre-commit passed. GLM-4.7-Flash with MTP3 and full decode graphs scored 52/64 GSM8K on TP2/PCP2/DP2 with EP and passed all nine stress requests. Non-EP TP1/PCP2/DP2 initially had one unparseable response; a diagnostic rerun scored 54/64 with zero invalid responses and passed all nine stress requests. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

mergify · 2026-09-16T16:59:52Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LucasWilkinson.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

github-actions · 2026-09-18T22:02:18Z

✅ Triggered Buildkite CI #89926 for commit 854c055d7cf2.

mergify · 2026-09-22T18:02:22Z

Hi @LucasWilkinson, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

LucasWilkinson · 2026-09-23T14:17:50Z

/ci run

github-actions · 2026-09-23T14:18:34Z

✅ Triggered Buildkite CI #90677 for commit ebbcc08b5059.

robertgshaw2-redhat · 2026-09-23T23:07:26Z

I have validated this on EP=32. The performance looks great

robertgshaw2-redhat · 2026-09-24T02:23:51Z

/ci retry

github-actions · 2026-09-24T02:24:11Z

✅ Triggered Buildkite CI #90834 for commit cb3595741a92, running 1 failed step(s) from Buildkite CI #90677.

robertgshaw2-redhat · 2026-09-24T02:32:32Z

https://buildkite.com/vllm/ci/builds/89926/list

last real run was AMD and unrelated kernel test - AMD and an unrelated B200 kernel test

Rebase onto main: main's PCP (vllm-project#56107), PCP+DP (vllm-project#57075) and MM-registry (vllm-project#57913) changes added pcp_manager/dp_sync/cudagraph_stats/aux_output_connector to the sample path and restructured test_mm_support_configured_after_model_load. Adapt the K=0 tests to the new runner surface; no production-code changes. Signed-off-by: Greg Weyer <gweyer@live.com>

* [Bugfix] Fix external LB DP rank handling when replicas share nodes (#53743) Signed-off-by: Tony Lin <tony.lin@intel.com> * [docs] Fix legacy hf CLI references (vllm) (#57958) Signed-off-by: Wauplin <lucainp@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * [Bugfix][NIXL] Fix DCP pulls across MLA cache regions (#57389) Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [ROCm] Refactor tuned gemms (#55001) Signed-off-by: Andy Friedrich <afriedri@amd.com> Signed-off-by: afriedri <afriedri@amd.com> Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com> Co-authored-by: Shanshan Shen <467638484@qq.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix] unskip InternViT test for transformers v5 compatibility (#55767) Signed-off-by: sahil <sahil@example.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> * [MM] Move get_dummy_processor_inputs into MM processor (#57967) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> * [ROCm][CI] Use ROCm backend for DeepSeek V4.1 ViT test (#57931) Signed-off-by: Djordje Ramic <djoramic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Feature] Add first-class KV hints request envelope for programmatic KV management (#53423) Signed-off-by: Karen Chung <karenc@nvidia.com> * [Docs] Add an Engram feature page explaining Engram usage in vLLM (#57910) Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai> * [ROCm][Perf] Route the fused shared-expert gate GEMM through the platform dispatcher (#54185) Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Engram] Drop redundant VLLM_PLE_CPU_OFFLOAD env var (#57937) Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai> * [Bugfix][MoE] Reject hash routing for unsupported monolithic backends (#57867) Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [Bugfix][ROCm] Reject unsupported EP for monolithic AITER MXFP4 MoE (#57866) Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [Docker] Use zstd for CI images and offer a Docker Hub variant (#55608) Signed-off-by: Nils Matteson <nilsmatteson@icloud.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Kimi-K3][AMD] Return KDA and MLA projection outputs directly (#50592) Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com> Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [CI][Build] Harden triton-cpu sleef submodule fetch in CPU image build (#57871) Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com> Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com> * [Bugfix][Kernel] Skip the fused silu-mul block-quant fast path when a swiglu clamp is set (#57984) Signed-off-by: Garrett Goon <garrett@primeintellect.ai> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * [Frontend] Add reusable TP1 initialized-engine snapshots (#51360) Signed-off-by: Nils Matteson <nilsmatteson@icloud.com> Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Pooling] MRV2 pooling shutdown model ref (#57737) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> * [CI] Split (H200) LM Eval Large Models into per-model jobs (#57965) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * [Scheduler] Soften Long Prefill Tokens Threshhold (#57951) Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> * [Perf][Attention] Reduce GLM sparse MLA preparation overhead (#57458) Signed-off-by: Summer Yang <girasoleyang@gmail.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> * [Bugfix] Annotate MTP draft KV cache groups positionally on the hybrid grouping path (#55390) Signed-off-by: Navjot Singh <navjot.singh@shopify.com> Co-authored-by: Codex <noreply@openai.com> * [Bugfix][GDN] Fix stateless first-chunk classification (#51565) Signed-off-by: taking-lying-flat <1615405@qq.com> Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Co-authored-by: zjy0516 <riverclouds.zhu@qq.com> * [CI] Add pre-commit check that new tests are tethered to Buildkite jobs (#54867) Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com> Signed-off-by: Turner <doubleujabbour@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Kevin H. Luu <khluu000@gmail.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [XPU] Fix Nemotron FP8 LM-eval config: drop CUDA-only moe_backend and wire to new Buildkite job (#49685) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * [Docker] Expose bundled vllm-rs on PATH (#57606) Signed-off-by: Alec Flowers <aflowers@nvidia.com> Signed-off-by: Alec <35311602+alec-flowers@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com> * [Fast Start] Cache the MTP draft model in a separate daemon group (#57312) Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Rust Frontend] Add MiMo V2.5 parser support (#57933) Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [Bugfix][Spec Decode] Cap DFlash/DSpark profiling query batch (#56448) Signed-off-by: wangyicong <wangyicong@bytedance.com> * [RL][Sleep] Retain frozen weights across level-2 sleep (#57891) Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: Codex <noreply@openai.com> * [ROCm][Bugfix] Explicitly reject FSE=1 with DPA+ETP deployment for DeepSeek-V4 (#57919) Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com> * [Spec Decode] Enable async scheduling for DFlash (#58065) * [Frontend][Rust] Add mm-processor benchmark for Rust frontend (#51922) Signed-off-by: Karthik Gangula <gangula-karthik@users.noreply.github.com> Signed-off-by: gangula-karthik <gkarthik923@gmail.com> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Karthik Gangula <gangula-karthik@users.noreply.github.com> Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Rust Frontend] Introduce parser-owned output grammar interfaces (#55269) Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [Security] Reject min_tokens that exceeds the filled max_tokens default (#57731) Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com> * [Bugfix][Frontend] Validate mixed prompt embedding mask lengths (#57006) Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com> Co-authored-by: Codex <noreply@openai.com> * [Bugfix][Qwen2.5-VL] Honor video fps for temporal M-RoPE (#47736) Signed-off-by: Ting Sun <suntcrick@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Rust Frontend] Build full-output grammars from initialized reasoning parsers (#57340) Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Codex <noreply@openai.com> * [ROCm][Perf] Avoid extra reshape kernel in Qwen GDN output norm (#47842) Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [Kernel] Add opt-in load-time MXFP4 dequantization (#50814) Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com> Co-authored-by: Shanshan Shen <467638484@qq.com> * [Rust Frontend] Separate multimodal instrumentation from request timing (#58084) Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [Kernel][DSV4.1] Fuse MXFP8 wo_b GEMM with sequence-parallel reduce-scatter (#57428) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Signed-off-by: Canlin <canlinguosdu@gmail.com> Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> * [CI] Emit a kernel symbol map from the csrc build (opt-in, for test selection) (#58097) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [perf] wire FA and FlashMLA for sm90 GLM5Next NoPE SparseMLA (#55385) Signed-off-by: JaredforReal <w13431838023@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> Co-authored-by: Leoyzen <leoyzen@gmail.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> * [CPU] Add device-memory-utilization CLI alias (#56547) Signed-off-by: louie-tsai <louie.tsai@intel.com> Signed-off-by: Louie Tsai <louie.tsai@intel.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * [Rust Frontend] Construct model-owned vision processors through specs (#58109) Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [BugFix][Core] Make the structured-output grammar poll non-blocking (#55931) Signed-off-by: ubwzwd <ubwzwd@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com> * [XPU][CI]Remove model_runner_v2 test from Intel GPU CI (#58050) Signed-off-by: zengxian <xiangdong.zeng@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [Bugfix][Structured Outputs] Reject empty `structural_tag` at request validation (#47450) Signed-off-by: linnea-lin-00638949 <15521435947@163.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com> * [Build] Fix CUDA 12 KV connector dependency selection (#57945) Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai> * [Bugfix][GLM-5.3-Flash] Run the dense MLP layers on the sequence-parallel shard (#58061) Signed-off-by: Jared Wen <w13431838023@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix][Structured Output] Disallow MRV1 + PP>1 + async sched + structured output (#56250) Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com> Co-authored-by: CNE Pierre FICHEPOIL <pierre-1.fichepoil@gendarmerie.interieur.gouv.fr> * [Feat][XPU] VLLM_BATCH_INVARIANT support for Dense/MoE models (#55881) Signed-off-by: Tony Lin <tony.lin@intel.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [SpecDecode] Restore residual-logits comments in _resample_kernel (#58166) Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [Bugfix] Narrow AuxOutput KV restrictions to known PD connectors (#58150) Signed-off-by: aoshen02 <aoshen@inferact.ai> * [Bugfix] prioritize architecture capability before DeepGEMM availability check (#58073) Signed-off-by: Tony Lin <tony.lin@intel.com> * [Bugfix][Attention] Avoid NaN in the Triton softcap for large attention logits (#56579) Signed-off-by: Kushal Dabbe <72650064+kushaldabbe@users.noreply.github.com> Co-authored-by: opencode <noreply@opencode.ai> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Bugfix][Engram] Fall back when /dev/shm is absent before sharing tables (#57914) Signed-off-by: Juntian Liu <juntianl@inferact.ai> Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix] hadacore_transform: respect inplace parameter to fix garbage outputs with QuIP transforms (#43462) Signed-off-by: Gilles Turpin <turpingilles15@gmail.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> * [Bugfix][ROCm] Dispatch the QuantFP8 CUDA fallback on the class (#58136) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][DSv4.1][Perf] Fuse the inverse RoPE into the sparse decode reduce (#57435) Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Fast Start] Support data parallelism in the weight cache daemon (#57386) Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Bugfix] batch_invariant: keep non-AllReduce collectives enabled on NCCL >= 2.31 (#58179) Signed-off-by: Guanxin Li <38149783+guanxingithub@users.noreply.github.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [MRV2] Release weight offloader on shutdown (#57834) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> * [Bugfix][V1] Honor enable_jit_warmup for V2 kernel warmup (#55146) Co-authored-by: mgoin <mgoin64@gmail.com> * [ROCm][CI] Add GELU activation for AiterExperts in the modular-kernel coverage (#58030) Signed-off-by: Divakar Verma <divakar.verma@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][V1] Read ModelState max_model_len from model config (#58149) Signed-off-by: Chenglun Hu <chenglunhu@gmail.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> * [Bugfix][Model][Spec Decode] Defer disposable GLM MTP head (#55442) Signed-off-by: Luca Motz <luca.motz@icloud.com> * [Refactor] Remove dead code multiple places (#58002) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Core] structured generation mode for DiffusionGemma model (Jev-like) (#57250) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Razorback16 <razorback16@protonmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Razorback16 <razorback16@protonmail.com> * [Kernel] Remove AllSpark INT8 W8A16 GEMM backend (#58001) Signed-off-by: mgoin <mgoin64@gmail.com> * [CI] Build the torch-nightly image on Ubuntu 24.04 (#58204) * [Bugfix] Backport Inductor custom-op pattern matching fix (#58189) Signed-off-by: mgoin <mgoin64@gmail.com> * [Core] Disable JIT warmup in eager mode (#58197) Signed-off-by: mgoin <mgoin64@gmail.com> * [CI] Split LM Eval TurboQuant KV Cache into per-config jobs (#57113) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Kimi <noreply@moonshot.ai> * [CI] Shard (H200 MIG 18GB) Spec Decode Draft Model across whole-directory replicas (#58193) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Perf] Remove CPU-GPU sync in heterogeneous vocabulary speculative decoding (#57396) * [ROCm][Build][The Rock] Bump Triton version to 3.8.x tip-of-tree with source build in The Rock image (#58006) Signed-off-by: Randall Smith <Randall.Smith@amd.com> * [Bugfix][ROCm] Use the platform FP8 range in the concat MLA q test (#58153) Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][Model][Bugfix] Enable GLM-5.2-MXFP4 on the deepseek_v32 path and fix sparse attention correctness (#51915) Signed-off-by: Jack Hu <Jack.Hu@amd.com> Signed-off-by: Jack Hu <jack.hu@amd.com> Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com> Co-authored-by: James E T Smith <jamesETsmith@users.noreply.github.com> Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com> Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com> * [ROCm] Use silu_and_mul_with_clamp's torch._C op (#52052) Signed-off-by: Tres Popp <tres.popp@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Shanshan Shen <467638484@qq.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix] Skip VllmConfig re-validation for with_hf_config submodel views (#58212) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.ai> Co-authored-by: Roger Wang <rogerw@inferact.ai> * [EPD] Support metadata-only audio inputs (#57887) Signed-off-by: Tianyu Guo <guoty@inferact.ai> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> * [ROCm][DSv4][Perf] Fuse the inverse RoPE into the sparse decode reduce (#57451) Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com> * [ROCm][Compile] Support BF16 AsyncTP fusion (#58098) Signed-off-by: Andreas Karatzas <akaratza@amd.com> * [CI][Bugfix] Update IPC test caller for #57312's _apply_entries signature (#58107) Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com> Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com> Co-authored-by: Kevin H. Luu <khluu000@gmail.com> * [ROCm][CI] Stage G gating (#50922) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [Bugfix] Set worker runtime threads before profiling and compilation (#55891) Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [XPU] Wire up SYCL apply_rotary_emb kernel in ApplyRotaryEmb (#55721) Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com> Signed-off-by: Michał Ganczarenko <michal.ganczarenko@intel.com> Co-authored-by: Claude <noreply@anthropic.com> * [Compilation] Fix QuTLASS compilation with PyTorch 2.13 (#58173) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Spec decode] Support variable-length decode for Kimi-K3 adaptive ver (#52988) Signed-off-by: Albert Cheng <albecheng@nvidia.com> Signed-off-by: Albert Cheng (Engrg-Hardware 1) <albecheng@login-bia01.bia.clusters.nvidia.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Benjamin Chislett <bchislett@nvidia.com> * [XPU][CI] Deselect tests/v1/spec_decode/test_mtp.py::test_glm_mtp_defers_lm_head (#58237) Signed-off-by: zengxian <xiangdong.zeng@intel.com> * [MoE] Use GateLinear for all MoE models (#58234) Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai> * [Bugfix][Frontend] Keep length finish_reason for max_tokens-truncated streaming tool calls (#46303) Signed-off-by: Ting Sun <suntcrick@gmail.com> * [ROCm][Perf] Use wvSplitK for single-output GEMMs (#53283) Signed-off-by: tangzzycc <3081129260@qq.com> * [Tests] Select V2 for diffusion scheduler unit tests (#58272) Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [Quantization] Select per-token NVFP4 MoE backends explicitly (#57176) Signed-off-by: aoshen02 <aoshen@inferact.ai> Signed-off-by: S1ro1 <matej.sirovatka@gmail.com> Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [ROCm][CI] Mirror the three TurboQuant evaluation groups on MI355 (#58282) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [ROCm][CI] Add MI355 dense NVFP4 and MoRI kernel mirrors (#58281) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [Mooncake] Address review nits from #56855 (#57174) Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai> Co-authored-by: Yifan Qiao <17067717+ivanium@users.noreply.github.com> * [Refactor][Quantization] Make FP8 and MLA weight transforms reusable pure functions (#57732) Signed-off-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: Yongye Zhu <zyy1102000@gmail.com> * [ROCm][Compile] Fuse AITER static FP8 attention output (#58099) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [ROCm][Bugfix] Register MRV2 sampler JIT warmups (#58092) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [Perf][Attention] Avoid CPU-GPU sync in DCP sequence lengths (#58169) Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [CI][Bugfix] Extend groupwise rms_norm scale tolerance to CUDA (#58252) Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com> Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com> * [Perf][ROCm][Attention] Narrow the Triton prefill-attention KV tile on RDNA3/RDNA4 (#58225) Signed-off-by: Jipeng Li <jipengli@amd.com> Co-authored-by: GitHub Copilot CLI <noreply@github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][Bugfix] Keep zero MiniMax MXFP8 activation blocks finite (#58089) Signed-off-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Include Python tooling in ROCm CI artifacts (#58271) Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [MRV2] Miscellaneous code cleanup (#57980) * [Bugfix][SM120][MLA] Support NoPE sparse MLA (GLM-5.3-Flash) on the FlashInfer SM120 backend (#55277) Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> * [XPU] upgrade to PyTorch 2.14 (#56013) Signed-off-by: Yan Ma <yan.ma@intel.com> Signed-off-by: Kunshang Ji <kunshang.ji@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [ROCm][Test] Check GDN prefill numerics and output ownership (#58091) Signed-off-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix] Disable prefix caching for encoder-only before model config hooks (#58287) Signed-off-by: Tianyu Guo <guoty@inferact.ai> * [Bugfix][Tool Parser] Migrate Granite to the streaming Parser Engine (#49648) Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com> * [DSpark] Support pipeline-parallel targets in aggregated serving (#56956) Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> Co-authored-by: Yongye Zhu <zyy1102000@gmail.com> * [Feature][Frontend] Add DeepSeek-V4 FIM completion rendering (#44229) Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com> * [Perf][MoE] Skip top-k slots routed to non-local experts in TritonExp… (#58051) Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com> Signed-off-by: Shuolei Wang <948904026@qq.com> * [CPU] Adds support for fp32 attention sinks (#56252) Signed-off-by: Ankit Jaiswal <ankit.jaiswal@amd.com> Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com> * [Quantization] Enable humming wNaM asymmetric quant (zero_point) with compressed-tensors (#46528) * [Quantization][Bugfix] Bump humming-kernels to 0.1.16 (#58054) Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Quark] Remove quark-specific silent online quantization (#51800) Signed-off-by: Felix Marty <Felix.Marty@amd.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> * [ROCm][Perf] Extend QK-norm/RoPE/KV-cache fusion to MRoPE (#50212) Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com> Co-authored-by: Santosh Hiremath <Santosh.Hiremath@amd.com> Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com> * [ROCm][CI] Validate Mooncake and NIXL prefill/decode accuracy (#58095) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [CI][ROCm] Add an MI355 Kimi-K3 unit test group (#58012) Signed-off-by: Oxana Korzh <okorzh@amd.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][NIXL] Restore successful push completion reporting (#58188) Signed-off-by: Dao Le <Dao007forever@gmail.com> Co-authored-by: Codex <noreply@openai.com> * [Bugfix][ROCm] Fix startup OOM in AITER MLA FP8 prefill workspace sizing (#57923) Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com> * Doc: add DiffusionGemma to supported models (#46466) Signed-off-by: Bruce <Bruce798858117@gmail.com> Signed-off-by: Misha Goin <mgoin64@gmail.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Perf] Use breakable CUDA graphs (no torch.compile) by default under VLLM_BATCH_INVARIANT so the tuned matmul configs see the runtime M (#57586) Signed-off-by: LioEinaudi <zhao3024667639@gmail.com> * [CI] Select one GPU for the H200 initialized snapshot E2E step (#58351) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Bugfix] Pick a KV block size supported by every attention backend (#49845) Signed-off-by: Divy <divy@coralbricks.ai> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * [Docs] Fix docstring typos (output_dytpe, kwrags, Abbrivations) (#55936) Signed-off-by: simpleqt <89645338+simpleqt@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [ROCm][Test] Cover MoRI graph replay and output lifetime (#58093) Signed-off-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][Bugfix] Fix TileLang mHC fused RMSNorm on 64-wide wavefronts (#58419) Signed-off-by: Djordje Ramic <djoramic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [CI][Bugfix] Limit MRV2 sampler JIT warmup registration to ROCm (#58465) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [CI] Disable JIT warmup by default in VllmRunner (#58452) Signed-off-by: mgoin <mgoin64@gmail.com> * [Bugfix][MRV2] Align dummy idx_mapping dtype to avoid runtime jit (#58462) Signed-off-by: Nick Hill <nickhill123@gmail.com> * [5/12][ci-selector][CI] Skip the Proton GPU test when another CUPTI tool is injected (#58455) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [CI] Share BF16 baselines across quantization comparison tests (#58469) Signed-off-by: Aarushi Jain <Aarushi.Jain2@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][DSA] Bound DeepSelect sentinel columns in the sparse top-k remap (#58215) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Test][Determinism] Cover chunked prefill in the batch-invariance suite (#55612) Signed-off-by: Bob Ok <49168652+blipbyte@users.noreply.github.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> * [CI][ROCM] Add the Fusion E2E TP2 Quick group on MI355, and the AITER MLA fix it needs (#58369) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: Andreas Karatzas <andreas.karatzas@protonmail.com> Co-authored-by: Codex <noreply@openai.com> * [Scheduler] Tune --long-prefill-token-threshold adaptiveness (#58459) Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> * [PCP] Support prefill context parallelism with data parallelism (#57075) Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: QiuChunshuo <qiuchunshuo@huawei.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [ROCm] Give turboquant boundary layers a layout-compatible backend (#54988) Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com> Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: Codex <noreply@openai.com> * [Kernel] Resubmit PR 48666 - Gemma4 FP8 KV FA4 head dim 512 backend selection (#53175) Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com> * [Bugfix][KV Offload] Retain offload event metadata through batch translation (#57453) Signed-off-by: Kapil Arya <kapila@nvidia.com> Signed-off-by: Kapil Arya <kapil.arya.17@gmail.com> Co-authored-by: Or Ozeri <or@ozery.com> * Fix full logprobs in token-in/token-out responses (#58488) Signed-off-by: aoshen02 <aoshen@inferact.ai> * [Bugfix][CPU][MoE] Fix out-of-bounds write and segfault when router weights are fp32 (#56168) Signed-off-by: Farzad Abdolhosseini <farzad@elastix.ai> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> * [Rust Frontend] Recognize new frontend-owned serve args as unsupported or no-op (#58330) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [Rust Frontend] Accept custom chat roles for HF templates (#58311) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [Dependency] Upgrade FlashInfer version to 0.7.0 (#58069) Signed-off-by: wzhao18 <wzhao18.sz@gmail.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Kevin H. Luu <khluu000@gmail.com> * [Bugfix][Spec Decode] Separate DSpark width from MTP stage validation (#54631) Signed-off-by: Luca Motz <luca.motz@icloud.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Rust Frontend] Pass vision preprocessing context for Nemotron-H (#57634) Pass the remaining engine context-length budget to model-owned vision processors through VisionPreprocessingContext. Preserve Nemotron batched engine fields and recognize llm_config as a text_config alias. Use the merged upstream llm-multimodal revision f0985ef65967615db2c79279aa07818499301bfd. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [Rust Frontend] Support `--sse-keep-alive-interval` (#58306) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [Compile][CI] Honor Triton cache overrides and add AMD timeout headroom (#58474) Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> Co-authored-by: Codex <noreply@openai.com> * [CPU][GDN] Support NIXL DS convolution-state layout (#53300) Signed-off-by: Li, Tianmu <tianmu.li@intel.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> * [ROCm][CI] Add the MI355 TurboQuant t3nc mirror (#58432) Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [EPD][Model Loader] Skip language-model checkpoint shards for `--mm-encoder-only` (#58086) Signed-off-by: grYe99 <guorongye99@gmail.com> Co-authored-by: grYe99 <guorongye99@gmail.com> Co-authored-by: Cursor Agent <cursoragent@cursor.com> * [Perf] Use Conv3dLayer for MiniMax M3 patch embedding (#58512) Signed-off-by: OpenAI Codex <codex@openai.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [Feature][Frontend] Request JSON body debug logging on `--enable-log-requests` flag (#58163) Signed-off-by: talora <talora@nvidia.com> * [CPU] Use pre-built triton (#58140) Signed-off-by: jiang1.li <jiang1.li@intel.com> * [Bugfix][V1] Reject encoder-cache hits with mismatched embedding counts (#57696) Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix] Capture prefill kernels for mixed FULL graphs (#58275) Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [Bugfix][CI] Fix the flaky sharded-sampling tests, and the engine teardown need (#58342) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][XPU] store the pointer raw bit pattern instead of its numeric value (#54514) Signed-off-by: Lai, Yejing <yejing.lai@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [Attention][CPU] Run Zen CPU encoder attention on zentorch SDPA (#54508) Signed-off-by: priyansh jain <priyansh.jain2@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [MRV2] Validate MRV2 entrypoint logits processors (#57728) Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> * [Bugfix][Rust Frontend] Prevent MM timing from enabling debug tracing (#58378) Co-authored-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: reidliu41 <reid201711@gmail.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [XPU][UT] Align HF and vLLM inputs for Qwen2 embedding test by preventing Sentence Transformers from applying chat template (#58117) Signed-off-by: RyanMa29 <ziyang.ma@intel.com> * [CPU] Gate the AVX10.2 paths on compiler support (#58133) Signed-off-by: R <Ganesh.R@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> * [Perf][Frontend] Offload streaming derender detokenization (#57528) Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com> * [Multimodal] Reuse the supplied tokenizer in the MiniMax-M3 VL processor (#58460) Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> * [XPU] enable XPU GRAPH by default (#51600) Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com> * [ROCm][DSv4.1][Perf] Emit MXFP8 from the sparse decode reduce and run wo_a as a grouped FP8 GEMM (#58456) Signed-off-by: fai <fangzhouai@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * Remove `.gemini/` and `CLAUDE.md` (#58541) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Bugfix][Pooling] Fix JinaVL label configuration and restore multimodal tests (#57347) Signed-off-by: Linze-Shi <linzeshi0@gmail.com> * [Chore] Use Transformers v5 names and drop redundant processor `use_fast` (#58550) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Refactor] Remove dead or duplicate tests (#58446) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Perf][Attention] Bound FlashInfer prefill dequantization scratch (#57918) Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com> * fix(config): apply presence_penalty/frequency_penalty from override-generation-config (#50769) Signed-off-by: Chenglun Hu <chenglunhu@gmail.com> Signed-off-by: hclsys <chenglunhu@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * [Bugfix] Resolve the Hub revision once per repo (#56092) Signed-off-by: Wauplin <lucainp@gmail.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Frontend] Remove the slow tokenizer mode (#58545) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Revert "[DSpark] Support pipeline-parallel targets in aggregated serving (#56956)" (#58484) * [transformer] RMSNorm matching for alternative rsqrt (#54461) Signed-off-by: Thomas Ortner <boh@zurich.ibm.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * [Bugfix] Count unsplit Idefics3 image patches (#48760) Signed-off-by: nightcityblade <nightcityblade@gmail.com> Co-authored-by: nightcityblade <nightcityblade@gmail.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> * [Bugfix] Keep JIT warmup under enforce-eager when fault tolerance is on (#58593) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Kimi <noreply@moonshot.cn> * [Core] Skip JIT monitor when JIT warmup is disabled (#58590) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> * [Fast Start] Wait for weight cache daemon readiness (#58370) * [Bugfix][Quantization] Add Humming to the W4A8 (INT4xFP8) MoE oracle (#58427) Signed-off-by: mgoin <mgoin64@gmail.com> * [Refactor] Move auxiliary files out of the repository root (#58572) Signed-off-by: mgoin <mgoin64@gmail.com> * [Cleanup] Remove online quantization support in `fp8.py` in favor of online shorthands (#53585) Signed-off-by: Felix Marty <Felix.Marty@amd.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [ROCm] Fix misrouting race-condition in multi-decode P/D disagg with mori-io (#51681) Signed-off-by: Vincent Cave <vincent.cave@amd.com> Signed-off-by: Shiksha Patel <shikpate@amd.com> Co-authored-by: Shiksha Patel <shikpate@amd.com> Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Perf] DiffusionGemma: constrained reads over the request's logprob_token_ids (#58216) Signed-off-by: Matt Mastracci <matthew@mastracci.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Bugfix] Pass quant_config to DiffusionGemma's ParallelLMHead (#48521) Signed-off-by: Aaron Kang <aaron.h.kang@icloud.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [ROCm][CI] skip the ROCm MRV1 default where MRV1 cannot serve the config (#58535) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [DFlash] Capture the context K/V precompute in the draft CUDA graph (#57632) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Bugfix][Outlines] Fix EOS termination and unconstrained masks after rejected drafts (#58612) * [Bugfix][KV Cache] Fix incremental multimodal block hashing (#51694) Signed-off-by: Jellow <49915976+CZT0@users.noreply.github.com> Signed-off-by: Jellow <dvdx@foxmail.com> * [XPU][CI] enable prompt embeds tests on XPU (#58283) Signed-off-by: Lin, Fanli <fanli.lin@intel.com> Signed-off-by: Fanli Lin <fanli.lin@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [CI] Report to CRCR after all jobs finish, gated on the build's long pole (#58628) * [PD][PushConnector] Record last activity of remotes on the D side (#52245) Signed-off-by: Sunita Nadampalli <nadampal@amazon.com> Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai> * [BUGFIX] fix ovis2_5 multimodal tokens (#52623) Signed-off-by: Milosz Grunwald <milosz.grunwald@intel.com> * [Bugfix][Core] Keep every multimodal feature in the partial-block KV event (#58288) Signed-off-by: haosenwang1018 <haosenwang1018@users.noreply.github.com> Co-authored-by: haosenwang1018 <haosenwang1018@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * [ROCm][CI] Mirror the DSv4-Flash disaggregated DP EP group on MI355 (#58558) Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix][Quantization] Give LM heads standard linear metadata (#58444) Signed-off-by: mgoin <mgoin64@gmail.com> * [Bugfix][Mamba] Restore prompt-tail prefix-cache hits with MTP (#58368) Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Benjamin Chislett <bchislett@nvidia.com> * [Perf] Parallelize registered CUDA Triton kernel warmup at startup (#58582) Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: Codex <noreply@openai.com> * [KV Connector] Fix DecodeBench fp8 fill values and add a startup fill mode (#58472) Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Release prompt_embeds tensor when its InputBatch slot is freed (#57988) Signed-off-by: khushali9 <khushali.desai9@gmail.com> * [Bugfix][KVConnector] Finalize saves on steps without a forward (#57775) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: Kimi Code <noreply@moonshot.ai> * [Bugfix][Frontend] Count reasoning tokens for Harmony, DeepSeek-V3 and Step3 parsers (#58626) Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [Bugfix] GLM-5.3-Flash: launch the kpool paged MQA logits in the varlen mode its schedule was built with (#55270) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [Bugfix] Accept EOS after grammar finish in outlines backend; reject json_object at validation (#57743) Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com> * [Perf] Batch Mamba2 prefill SSM state saves, removing GPU<->CPU syncs (#49371) Signed-off-by: samuelkim7 <samuelmwkim@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> * [CI] Run DFlash2 NVFP4 acceptance test on B200; skip it on H200 35GB MIG (#58496) Signed-off-by: khluu <khluu000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Misha Goin <mgoin64@gmail.com> * [Bugfix][MRV2] Treat padded prompt tails as spec-decode rows for hybrid models (#58434) Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Kimi-K3][Perf] Dispatch GEMM for vision patch embedder (#58527) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> * [Minimax-M3][Perf] Use triton_mrope for vision tower + int64 offset fix for triton_mrope (#58526) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: Kimi <noreply@moonshot.cn> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Bugfix][Quantization] Refresh online NVFP4 scales before reload post-processing (#57954) Signed-off-by: S1ro1 <matej.sirovatka@gmail.com> Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: aoshen02 <aoshen@inferact.ai> * [Perf][DSv4.1] Restore the fused query RMSNorm + MXFP8 quantization path (#57679) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> * [Bugfix][LogitsProcessor] Validate ':' separator in custom logits processor FQCN (#56020) Signed-off-by: 100milliongold <gadian88@gmail.com> * [ROCm][CI][AITER Coverage] Harden MoE sorting-backend/dispatch env-var test matrix (#58393) Signed-off-by: Divakar Verma <divakar.verma@amd.com> * [gRPC] Fix ping tolerance so long non-streaming RPCs are not dropped (#55102) Signed-off-by: Wei Gong <wei@together.ai> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> * [Perf][Rust Frontend] Make histogram observations lock-free (#58574) Co-authored-by: jthomson04 <jwillthomson19@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [ROCm][Perf] MXFP8 GEMM on native 32x32 block scales for gfx950 (#58510) Signed-off-by: fai <fangzhouai@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Bugfix][Qwen4Exp] Keep pinned PLE prefetch ids out of the CUDA graph pool (#58489) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Perf][Engram] Serialize offloaded lookups and pack host tables into huge pages (#56926) Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix][Quantization] Fix MXFP8 startup crash on layers below mm_mxfp8 shape limits (#54223) Signed-off-by: samuelkim7 <samuelmwkim@gmail.com> Signed-off-by: mgoin <mgoin64@gmail.com> Co-authored-by: mgoin <mgoin64@gmail.com> * [ROCm] Cut 69 wasted contiguous copies per decode step from the skinny GEMM path (#58566) Signed-off-by: lifulu <fululi12@amd.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * [ROCm][Build] Filter crate tags from vLLM version detection (#57744) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: reidliu41 <reid201711@gmail.com> * [Perf][Distributed] Add low-SM multimem reduce-scatter for SM100/SM103 (#55072) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Signed-off-by: Summer Yang <girasoleyang@gmail.com> Co-authored-by: Summer Yang <girasoleyang@gmail.com> * [PP][XPU]Add the flag to control microbatch feature on MRV2+PP (#55145) Signed-off-by: yisheng <yi.sheng@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [Feature] Triton kernel dispatcher (#43048) Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> * [ROCm] Fix CI runtime and tests for MI355 DPX (#58244) Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Mahesh Kunreddi <mahesh.kunreddi@amd.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [CI][ROCm] Prevent Model Executor apt stalls (#58607) Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: Codex <noreply@openai.com> * [Qwen4Exp][ROCm] PLE n-gram table CPU offload (#57497) Signed-off-by: Mathew Odden <modden@redhat.com> Signed-off-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: opencode+deepseek-v4-flash+vllm <opencode+deepseek-v4-flash+vllm@example.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: OpenAI Codex <noreply@openai.com> * [MM] Add Triton kernel for mm_input_normal. (#56798) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <Isotr0py@outlook.com> * [Structured Outputs] Parse Lark grammars natively in the xgrammar backend (#58321) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [ROCm] Credit ROCm/aiter for the block32 GEMM's packed kernel and in-launch split-K (#58659) Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> * [Frontend] Handle Disable Thinking in /v1/messages (#58613) Signed-off-by: jryberg <johan.ryberg@security.ntt> Signed-off-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: jryberg <johan.ryberg@security.ntt> Co-authored-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [Bugfix] Stop leaking the internal field name in the max_tokens validation error (#58336) Signed-off-by: shallow10 <495593563@qq.com> * [Bugfix][KV Cache][MLA] Align packed block strides for V3.2 sparse MLA (#55528) Signed-off-by: lz <145014769+200lz@users.noreply.github.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Perf][DSv4] Fuse inverse RoPE + FP8 quant into FlashInfer sparse MLA (#58621) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [UX][Frontend] Introduce `vllm preload` cli for fast restart (#56680) Signed-off-by: Isotr0py <Isotr0py@outlook.com> * [MoE] Defer the TRTLLM-Gen top-k finalize on the modular path (#58635) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Docs] Add return annotation to `fused_mm_input_norm_triton` (#58687) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [CI] Shard (H100) Helion Kernels five ways (#58645) Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> * [Core] Model console logging as CLI configuration (#57205) Add `--logging-config` CLI argument which can be supplied as JSON or using dotted arguments. The `--log-level` argument is provided for convenience, and `--log-config-file` is deprecated in favor of `--logging-config.pylogging_config_file`. Signed-off-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: AI Assistant <noreply@openai.com> * [ROCm][CI] Pass weight_shape in MXFP8 block32 linear tests (#58698) Signed-off-by: Djordje Ramic <djoramic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> --------- Signed-off-by: Tony Lin <tony.lin@intel.com> Signed-off-by: Wauplin <lucainp@gmail.com> Signed-off-by: Lucas Wilkinson <lwilkinson@neuralmagic.com> Signed-off-by: Andy Friedrich <afriedri@amd.com> Signed-off-by: afriedri <afriedri@amd.com> Signed-off-by: sahil <sahil@example.com> Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk> Signed-off-by: Djordje Ramic <djoramic@amd.com> Signed-off-by: Karen Chung <karenc@nvidia.com> Signed-off-by: NickLucche <nicolo.lucchesi@mistral.ai> Signed-off-by: Mikko Tukiainen <Mikko.Tukiainen@amd.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Signed-off-by: Nils Matteson <nilsmatteson@icloud.com> Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com> Signed-off-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com> Signed-off-by: Garrett Goon <garrett@primeintellect.ai> Signed-off-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Signed-off-by: Thang Nguyen <thangnguyenvn647@gmail.com> Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Signed-off-by: Robert Shaw <robshaw@redhat.com> Signed-off-by: Summer Yang <girasoleyang@gmail.com> Signed-off-by: Navjot Singh <navjot.singh@shopify.com> Signed-off-by: taking-lying-flat <1615405@qq.com> Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Signed-off-by: Turner Jabbour <doubleujabbour@gmail.com> Signed-off-by: Turner <doubleujabbour@gmail.com> Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com> Signed-off-by: Alec Flowers <aflowers@nvidia.com> Signed-off-by: Alec <35311602+alec-flowers@users.noreply.github.com> Signed-off-by: liusy58 <mg21330037@smail.nju.edu.cn> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> Signed-off-by: wangyicong <wangyicong@bytedance.com> Signed-off-by: aoshen02 <aoshen02@users.noreply.github.com> Signed-off-by: Shanshan Shen <87969357+shen-shanshan@users.noreply.github.com> Signed-off-by: Karthik Gangula <gangula-karthik@users.noreply.github.com> Signed-off-by: gangula-karthik <gkarthik923@gmail.com> Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com> Signed-off-by: 子华 <huaxi.shx@alibaba-inc.com> Signed-off-by: Ting Sun <suntcrick@gmail.com> Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Signed-off-by: Canlin <canlinguosdu@gmail.com> Signed-off-by: khluu <khluu000@gmail.com> Signed-off-by: JaredforReal <w13431838023@gmail.com> Signed-off-by: louie-tsai <louie.tsai@intel.com> Signed-off-by: Louie Tsai <louie.tsai@intel.com> Signed-off-by: ubwzwd <ubwzwd@gmail.com> Signed-off-by: zengxian <xiangdong.zeng@intel.com> Signed-off-by: linnea-lin-00638949 <15521435947@163.com> Signed-off-by: Jee Jee Li <jeejeelee@inferact.ai> Signed-off-by: Jared Wen <w13431838023@gmail.com> Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com> Signed-off-by: aoshen02 <aoshen@inferact.ai> Signed-off-by: Kushal Dabbe <72650064+kushaldabbe@users.noreply.github.com> Signed-off-by: Juntian Liu <juntianl@inferact.ai> Signed-off-by: Juntian Liu <Juntianl777@gmail.com> Signed-off-by: Gilles Turpin <turpingilles15@gmail.com> Signed-off-by: Stefan Koncarevic <Stefan.Koncarevic@amd.com> Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com> Signed-off-by: Guanxin Li <38149783+guanxingithub@users.noreply.github.com> Signed-off-by: Divakar Verma <divakar.verma@amd.com> Signed-off-by: Chenglun Hu <chenglunhu@gmail.com> Signed-off-by: mgoin <mgoin64@gmail.com> Signed-off-by: Luca Motz <luca.motz@icloud.com> Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Matt Mastracci <matthew@mastracci.com> Signed-off-by: Razorback16 <razorback16@protonmail.com> Signed-off-by: Randall Smith <Randall.Smith@amd.com> Signed-off-by: Jack Hu <Jack.Hu@amd.com> Signed-off-by: Jack Hu <jack.hu@amd.com> Signed-off-by: Douglas Lehr <Doug.Lehr@amd.com> Signed-off-by: Tres Popp <tres.popp@amd.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Signed-off-by: Tianyu Guo <guoty@inferact.ai> Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Andreas Karatzas <andreas.karatzas@protonmail.com> Signed-off-by: Michal Ganczarenko <michal.ganczarenko@intel.com> Signed-off-by: Michał Ganczarenko <michal.ganczarenko@intel.com> Signed-off-by: Albert Cheng <albecheng@nvidia.com> Signed-off-by: Albert Cheng (Engrg-Hardware 1) <albecheng@login-bia01.bia.clusters.nvidia.com> Signed-off-by: tangzzycc <3081129260@qq.com> Signed-off-by: S1ro1 <matej.sirovatka@gmail.com> Signed-off-by: Jipeng Li <jipengli@amd.com> Signed-off-by: Zihua Wu <13583761+lucifer1004@users.noreply.github.com> Signed-off-by: Yan Ma <yan.ma@intel.com> Signed-off-by: Kunshang Ji <kunshang.ji@intel.com> Signed-off-by: Nikhil Kulkarni <nikhilkulkarni1755@gmail.com> Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: Shuolei Wang <shuoleiwang123@gmail.com> Signed-off-by: Shuolei Wang <948904026@qq.com> Signed-off-by: Ankit Jaiswal <ankit.jaiswal@amd.com> Signed-off-by: jinzhen.ljz <jinzhen.ljz@antgroup.com> Signed-off-by: Felix Marty <Felix.Marty@amd.com> Signed-off-by: Vorapol Assavasangthong <Vorapol.Assavasangthong@amd.com> Signed-off-by: Oxana Korzh <okorzh@amd.com> Signed-off-by: Dao Le <Dao007forever@gmail.com> Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com> Signed-off-by: Bruce <Bruce798858117@gmail.com> Signed-off-by: Misha Goin <mgoin64@gmail.com> Signed-off-by: LioEinaudi <zhao3024667639@gmail.com> Signed-off-by: Divy <divy@coralbricks.ai> Signed-off-by: simpleqt <89645338+simpleqt@users.noreply.github.com> Signed-off-by: Aarushi Jain <Aarushi.Jain2@amd.com> Signed-off-by: Bob Ok <49168652+blipbyte@users.noreply.github.com> Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com> Signed-off-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Signed-off-by: Stefan Koncarevic <stefan.koncarevic@amd.com> Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com> Signed-off-by: Kapil Arya <kapila@nvidia.com> Signed-off-by: Kapil Arya <kapil.arya.17@gmail.com> Signed-off-by: Farzad Abdolhosseini <farzad@elastix.ai> Signed-off-by: wzhao18 <wzhao18.sz@gmail.com> Signed-off-by: Li, Tianmu <tianmu.li@intel.com> Signed-off-by: grYe99 <guorongye99@gmail.com> Signed-off-by: OpenAI Codex <codex@openai.com> Signed-off-by: talora <talora@nvidia.com> Signed-off-by: jiang1.li <jiang1.li@intel.com> Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com> Signed-off-by: Lai, Yejing <yejing.lai@intel.com> Signed-off-by: priyansh jain <priyansh.jain2@amd.com> Signed-off-by: reidliu41 <reid201711@gmail.com> Signed-off-by: RyanMa29 <ziyang.ma@intel.com> Signed-off-by: R <Ganesh.R@amd.com> Signed-off-by: Shrey Gajjar <shreygajjar007@gmail.com> Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com> Signed-off-by: fai <fangzhouai@gmail.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Signed-off-by: Linze-Shi <linzeshi0@gmail.com> Signed-off-by: Yueh-Ting Chen <yueh.ting.chen@gmail.com> Signed-off-by: hclsys <chenglunhu@gmail.com> Signed-off-by: Thomas Ortner <boh@zurich.ibm.com> Signed-off-by: nightcityblade <nightcityblade@gmail.com> Signed-off-by: Vincent Cave <vincent.cave@amd.com> Signed-off-by: Shiksha Patel <shikpate@amd.com> Signed-off-by: Aaron Kang <aaron.h.kang@icloud.com> Signed-off-by: Jellow <49915976+CZT0@users.noreply.github.com> Signed-off-by: Jellow <dvdx@foxmail.com> Signed-off-by: Lin, Fanli <fanli.lin@intel.com> Signed-off-by: Fanli Lin <fanli.lin@intel.com> Signed-off-by: Sunita Nadampalli <nadampal@amazon.com> Signed-off-by: Milosz Grunwald <milosz.grunwald@intel.com> Signed-off-by: haosenwang1018 <haosenwang1018@users.noreply.github.com> Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Signed-off-by: khushali9 <khushali.desai9@gmail.com> Signed-off-by: Samyabrata Maji <116789799+sammaji@users.noreply.github.com> Signed-off-by: SIDDARTHA REDDY <75976672+SIDDARTHAREDDY8@users.noreply.github.com> Signed-off-by: samuelkim7 <samuelmwkim@gmail.com> Signed-off-by: 100milliongold <gadian88@gmail.com> Signed-off-by: Wei Gong <wei@together.ai> Signed-off-by: lifulu <fululi12@amd.com> Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Signed-off-by: yisheng <yi.sheng@intel.com> Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com> Signed-off-by: Mathew Odden <modden@redhat.com> Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Signed-off-by: Lingpeng Jin <103567126+valarLip@users.noreply.github.com> Signed-off-by: jryberg <johan.ryberg@security.ntt> Signed-off-by: shallow10 <495593563@qq.com> Signed-off-by: lz <145014769+200lz@users.noreply.github.com> Signed-off-by: Mark McLoughlin <markmc@redhat.com> Co-authored-by: Tony Lin <tony.lin@intel.com> Co-authored-by: Lucain <lucainp@gmail.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com> Co-authored-by: OpenAI Codex <noreply@openai.com> Co-authored-by: afriedri <afriedri@amd.com> Co-authored-by: Douglas Lehr <91553416+dllehr-amd@users.noreply.github.com> Co-authored-by: Shanshan Shen <467638484@qq.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Sahil Patel <91423311+Sip4818@users.noreply.github.com> Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk> Co-authored-by: djramic <djoramic@amd.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: Karen Chung <karenc@nvidia.com> Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@mistral.ai> Co-authored-by: Mikko Tukiainen <mikko.tukiainen@amd.com> Co-authored-by: Nils Matteson <nilsmatteson@icloud.com> Co-authored-by: yinfengLiu <yinfeliu@amd.com> Co-authored-by: Liuyinfeng01 <199041580+LiuYinfeng01@users.noreply.github.com> Co-authored-by: vllm-agent <claw@inferact.ai> Co-authored-by: kimi-no-na-wa <kimi-no-na-wa@users.noreply.github.com> Co-authored-by: Garrett Goon <44747910+garrett361@users.noreply.github.com> Co-authored-by: Taneem Ibrahim <taneem.ibrahim@gmail.com> Co-authored-by: elehayym <52448798+Yuzu23@users.noreply.github.com> Co-authored-by: Thang Nguyen <69278249+Thangnguyenvn98@users.noreply.github.com> Co-authored-by: Kimi Code <noreply@moonshot.cn> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: Robert Shaw <robshaw@redhat.com> Co-authored-by: Summer Yang <girasoleyang@gmail.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Co-authored-by: Navjot Singh <navjot.singh@uwaterloo.ca> Co-authored-by: cherry77-cloud <1615405@qq.com> Co-authored-by: Turner Jabbour <doubleujabbour@gmail.com> Co-authored-by: Kevin H. Luu <khluu000@gmail.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Chaojun Zhang <chaojun.zhang@intel.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Alec <35311602+alec-flowers@users.noreply.github.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: siyu <mg21330037@smail.nju.edu.cn> Co-authored-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Yicong Wang <wangyicong@bytedance.com> Co-authored-by: aoshen02 <aoshen@inferact.ai> Co-authored-by: aoshen02 <aoshen02@users.noreply.github.com> Co-authored-by: Jeff (Junze) Ma <93145857+majunze2001@users.noreply.github.com> Co-authored-by: karthik <56480632+gangula-karthik@users.noreply.github.com> Co-authored-by: Karthik Gangula <gangula-karthik@users.noreply.github.com> Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> Co-authored-by: Juan Pérez de Algaba <124347725+jperezdealgaba@users.noreply.github.com> Co-authored-by: shaohuaxi <huaxi.shx@alibaba-inc.com> Co-authored-by: Ting SUN <suntcrick@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Canlin Guo <canlinguosdu@gmail.com> Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Jared Wen <w13431838023@gmail.com> Co-authored-by: Leoyzen <leoyzen@gmail.com> Co-authored-by: Louie Tsai <louie.tsai@intel.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: ubwzwd <ubwzwd@gmail.com> Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com> Co-authored-by: xiangdong <40376367+zxd1997066@users.noreply.github.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> Co-authored-by: linyafeng <15521435947@163.com> Co-authored-by: CNE Pierre FICHEPOIL <pierre-1.fichepoil@gendarmerie.interieur.gouv.fr> Co-authored-by: Flora Feng <4florafeng@gmail.com> Co-authored-by: Kushal <72650064+kushaldabbe@users.noreply.github.com> Co-authored-by: opencode <noreply@opencode.ai> Co-authored-by: Misha Goin <mgoin64@gmail.com> Co-authored-by: Juntian Liu <Juntianl777@gmail.com> Co-authored-by: Gilles Turpin <turpingilles15@gmail.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: stefankoncarevic <Stefan.Koncarevic@amd.com> Co-authored-by: Fangzhou Ai <31551580+Fangzhou-Ai@users.noreply.github.com> Co-authored-by: Guanxin Li <38149783+guanxingithub@users.noreply.github.com> Co-authored-by: pengyihang <1017861497@qq.com> Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com> Co-authored-by: hcl <chenglunhu@gmail.com> Co-authored-by: lucamotz <luca.motz@icloud.com> Co-authored-by: Matt Mastracci <matthew@mastracci.com> Co-authored-by: Lucas Wilkinson <lwilkins@redhat.com> Co-authored-by: Razorback16 <razorback16@protonmail.com> Co-authored-by: Andrey Talman <atalman@fb.com> Co-authored-by: Kimi <noreply@moonshot.ai> Co-authored-by: Michael Lapshin <55516685+MichaelLapshin@users.noreply.github.com> Co-authored-by: rasmith <Randall.Smith@amd.com> Co-authored-by: Jack Hu <jack.hu@amd.com> Co-authored-by: James E T Smith <jamesETsmith@users.noreply.github.com> Co-authored-by: Douglas Lehr <Doug.Lehr@amd.com> Co-authored-by: Tres <tpopp@users.noreply.github.com> Co-authored-by: Roger Wang <rogerw@inferact.ai> Co-authored-by: Tianyu Guo <guoty@inferact.ai> Co-authored-by: Michał Ganczarenko <michal.gancz…

pisceskkk added 7 commits September 15, 2026 16:49

Support prefill context parallelism with data parallelism

348748a

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>

Use a combined MoE DP-PCP communication group

10f43e9

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>

Use the MoE DP-PCP group for static non-SP dispatch

4ed8abf

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>

Route non-SP MoE through the DP-PCP group

0c3b063

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>

Coordinate MoE token counts across DP and PCP

3cc259b

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>

[MoE] Rename non-SP communication metadata

0696b03

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>

[MoE] Preserve AgRs dispatch statement order

168e2b9

Signed-off-by: QiuChunshuo <qiuchunshuo@huawei.com>

mergify Bot added deepseek Related to DeepSeek models nvidia labels Sep 15, 2026

github-project-automation Bot added this to NVIDIA Sep 15, 2026

mergify Bot added the rocm Related to AMD ROCm label Sep 15, 2026

github-project-automation Bot added this to AMD Sep 15, 2026

mergify Bot added the speculative-decoding label Sep 15, 2026

github-project-automation Bot moved this to Todo in AMD Sep 15, 2026

mergify Bot added the mrv2 Model Runner V2 specific label Sep 15, 2026

LucasWilkinson and others added 6 commits September 16, 2026 01:34

LucasWilkinson force-pushed the pcp-dp branch from 5bb5e22 to de81d52 Compare September 16, 2026 05:34

LucasWilkinson and others added 4 commits September 16, 2026 15:31

mergify Bot added the glm label Sep 16, 2026

mergify Bot added the needs-rebase label Sep 16, 2026

LucasWilkinson changed the title ~~[Core][PCP] Support prefill context parallelism with data parallelism (continues #54131)~~ [PCP] Support prefill context parallelism with data parallelism (continues #54131) Sep 16, 2026

LucasWilkinson enabled auto-merge (squash) September 18, 2026 22:02

LucasWilkinson and others added 3 commits September 21, 2026 10:29

Merge branch 'main' into pcp-dp

03afdaf

Merge branch 'main' into pcp-dp

22947aa

Merge branch 'main' into pcp-dp

cff6374

[CI] Run tests/test_pcp_dp.py in CPU job

cbdaef5

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

LucasWilkinson requested review from Harry-Chen and khluu as code owners September 23, 2026 03:07

mergify Bot added the ci/build label Sep 23, 2026

Merge branch 'main' into pcp-dp

2765c63

robertgshaw2-redhat mentioned this pull request Sep 23, 2026

[Bug] ep_gather output store overflows int32 with DeepEP v2 expanded layout (IMA in _fwd_kernel_ep_gather) #58365

Open

Merge branch 'main' into pcp-dp

ebbcc08

Merge branch 'main' into pcp-dp

cc6cf84

robertgshaw2-redhat disabled auto-merge September 23, 2026 23:06

Merge branch 'main' into pcp-dp

57a7d05

robertgshaw2-redhat approved these changes Sep 23, 2026

View reviewed changes

robertgshaw2-redhat enabled auto-merge (squash) September 23, 2026 23:07

Merge branch 'main' into pcp-dp

cb35957

robertgshaw2-redhat disabled auto-merge September 24, 2026 02:31

robertgshaw2-redhat merged commit e6dc16c into vllm-project:main Sep 24, 2026
8 of 11 checks passed

gf239 mentioned this pull request Sep 29, 2026

[Perf][Attention] FlashInfer: skip the seq_lens copy with speculative decoding #57352

Open

3 of 4 tasks

Navigation Menu

Repository navigation

[PCP] Support prefill context parallelism with data parallelism - #57075

[PCP] Support prefill context parallelism with data parallelism#57075
robertgshaw2-redhat merged 34 commits into
vllm-project:mainfrom
LucasWilkinson:pcp-dp

LucasWilkinson commented Sep 15, 2026 •

edited

Loading

Uh oh!

mergify Bot commented Sep 16, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

mergify Bot commented Sep 22, 2026

Uh oh!

LucasWilkinson commented Sep 23, 2026

Uh oh!

github-actions Bot commented Sep 23, 2026

Uh oh!

robertgshaw2-redhat commented Sep 23, 2026

Uh oh!

robertgshaw2-redhat commented Sep 24, 2026

Uh oh!

github-actions Bot commented Sep 24, 2026

Uh oh!

robertgshaw2-redhat commented Sep 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

LucasWilkinson commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify Bot commented Sep 16, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

mergify Bot commented Sep 22, 2026

Uh oh!

LucasWilkinson commented Sep 23, 2026

Uh oh!

github-actions Bot commented Sep 23, 2026

Uh oh!

robertgshaw2-redhat commented Sep 23, 2026

Uh oh!

robertgshaw2-redhat commented Sep 24, 2026

Uh oh!

github-actions Bot commented Sep 24, 2026

Uh oh!

robertgshaw2-redhat commented Sep 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

LucasWilkinson commented Sep 15, 2026 •

edited

Loading