Repository navigation
[ROCm][CI] Introducing new MI300 nodes - #39531
Merged
Merged
Conversation
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces an auto-generated catalog of single-GPU tests for the mi300_8 agent pool in the AMD Buildkite configuration. The review feedback identifies several test steps that incorrectly request or imply multi-GPU requirements (such as num_gpus: 2 or tensor parallelism tests), which are incompatible with the single-GPU mi300_8 queue. These misconfigured steps should be removed to ensure CI stability.
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
AndreasKaratzas
marked this pull request as ready for review
April 10, 2026 20:35
AndreasKaratzas
added
the
ready
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
AndreasKaratzas
changed the title
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
tjtanaa
reviewed
Apr 20, 2026
|
|
||
| - label: Kernels FP8 MoE Test # TBD | ||
| - label: Quantized MoE Test (B200-MI300) # TBD |
Member
There was a problem hiding this comment.
@AndreasKaratzas can you help to look through the test so that there are no test cases that are doing nothing? This is to cutdown the resource usage.
Quantized MoE Test (B200-MI300) doesn't seem to run any tests.
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
AndreasKaratzas
requested review from
mgoin,
pavanimajety,
robertgshaw2-redhat and
yewentao256
as code owners
April 20, 2026 05:31
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
tjtanaa
enabled auto-merge (squash)
April 20, 2026 08:08
tjtanaa
disabled auto-merge
April 20, 2026 08:09
avinashsingh77
pushed a commit
to avinashsingh77/vllm
that referenced
this pull request
Apr 27, 2026
Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: Avinash Singh <avinashsingh.rcoem@gmail.com>
weifang231
pushed a commit
to weifang231/eb-vllm
that referenced
this pull request
May 13, 2026
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
my-other-github-account
pushed a commit
to my-other-github-account/vllm
that referenced
this pull request
May 15, 2026
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
zero-uuuuk
added a commit
to zero-uuuuk/vllm
that referenced
this pull request
Oct 4, 2026
* fix: Do not make function calls when request has no tools for /v1/responses (#40314) Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> * [Qwen][Bugfix] Fixes sigmoid activation in torch impl of RMSNormGated. (#40245) Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com> * [XPU] [torch.compile] Skipping CUDA graph memory estimation to avoid startup errors. (#39977) Signed-off-by: chaojun-zhang <chaojun.zhang@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [BUGFIX] Fix Pixtral consolidated format vision weight loading (#39916) Signed-off-by: Julien Denize <julien.denize@mistral.ai> Signed-off-by: juliendenize <julien.denize@mistral.ai> * [CPU][RISC-V] Support multiple RVV VLEN targets via compile-time dispatch (#39478) Signed-off-by: velonica0 <like@mail.nankai.edu.cn> * [Fix] Add Spacing when Requesting Output Token > max_model_len (#40324) Signed-off-by: San-Nguyen <san.nguyen@ibm.com> * [Bugfix] Forward mm_processor_kwargs in offline generate APIs (#40251) Signed-off-by: wuyingjun <wuyingjun_yewu@cmss.chinamobile.com> * [Docs] Fix thinking_token_budget docs (#40316) Signed-off-by: milesial <milesial@users.noreply.github.com> * [ROCm][CI] Introducing new MI300 nodes (#39531) Signed-off-by: Andreas Karatzas <akaratza@amd.com> * [Bugfix] Fix scaled_mm output narrowing for 3D input tensors (#38093) Signed-off-by: nemanjaudovic <nudovic@amd.com> * [CPU][BugFix] Fix inter-node pipeline parallel (#40150) Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com> * [kv_offload+HMA][4/N]: Support sliding window lookup (#36645) Signed-off-by: Or Ozeri <oro@il.ibm.com> Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com> * [EPLB] Add nixl-based eplb communicator (#36276) Signed-off-by: ilmarkov <markovilya197@gmail.com> Signed-off-by: Markov Ilya <markovilya19@gmail.com> * nixl refactor [2/N]: unify TpKVTopology + HeteroTPTransferConfig into TransferTopology (#39529) Signed-off-by: Zhanqiu Hu <zhu@redhat.com> * [XPU] enable triton attention test on XPU by removing cuda device binding (#39627) Signed-off-by: Yan Ma <yan.ma@intel.com> * [Anthropic][Frontend] Added chat_template_kwargs to /v1/messages (#40125) Signed-off-by: Aleksandar Yanakiev <alexander.yanakiev@discretestack.com> Co-authored-by: Aleksandar Yanakiev <alexander.yanakiev@discretestack.com> * [bugfix] Use only onlines CPUs in lscpu (#40161) Signed-off-by: kse <kevin.sejourne@cloud-temple.com> Co-authored-by: kse <kevin.sejourne@cloud-temple.com> * mxfp8 online quant move to new frontend (#40152) Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com> * [CI][EPLB] Add Async EPLB end-to-end integration test to CI (#40168) Signed-off-by: Sage Moore <sage@neuralmagic.com> * [ROCm][Feature] Enable AITER MLA attention backend to work with Eagle3 speculative decoding on ROCm (#39616) Signed-off-by: larryli2-amd <larryli2@amd.com> Co-authored-by: TJian <tunjian.tan@embeddedllm.com> * [Bug] Fix dcp error message (#40053) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [ROCm] Add MLA dual RMS norm fusion (Q, KV) pass for DeepSeek/Kimi-K2 (#39242) Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com> * Properly enable wvSplitK fp8 path for RDNA (#37712) Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com> * [XPU] fix MoE triton backend in online fp8 quantization (#40109) Signed-off-by: Yan Ma <yan.ma@intel.com> * [MoE Refactor] Move the shared/fused expert output sum into MoERunnerBase (#35949) Signed-off-by: Bill Nell <bnell@redhat.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> * [EPLB] Refactor Async EPLB synchronization logic (#37601) Signed-off-by: Sage Moore <sage@neuralmagic.com> Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com> * [Core] Label torch trace logging overhead with dynamo_timed (#39329) Signed-off-by: Frederik Gossen <frgossen@meta.com> * Update flashinfer to 0.6.8 (#39959) Signed-off-by: bai <v@gor.io> * [Core] Pass donate_graph_module=True to standalone_compile (#39733) Signed-off-by: Frederik Gossen <frgossen@meta.com> * [WideEP] Remove naive all2all. Use allgather_reducescatter instead (#33728) Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com> * [Bugfix][CI] Fix `tests/distributed/test_torchrun_example_moe.py` (#40349) Signed-off-by: NickLucche <nlucches@redhat.com> * [Core] Cache InductorPass.hash_source with functools.cache (#39328) Signed-off-by: Frederik Gossen <frgossen@meta.com> * Enable building MoRI with AMD AINIC stack (#38371) Signed-off-by: Theresa Shan <thshan@smci355-ccs-aus-n08-21.prov.aus.ccs.cpe.ice.amd.com> Signed-off-by: Theresa Shan <theresa.shan@amd.com> Co-authored-by: Theresa Shan <thshan@smci355-ccs-aus-n08-21.prov.aus.ccs.cpe.ice.amd.com> Co-authored-by: TJian <tunjian.tan@embeddedllm.com> * [LMCache MP Connector] Add num_lmcache_extra_cached_token in KVTransferParams (#39843) Signed-off-by: aeon-x <talexcao@gmail.com> * [EPLB] Consolidate is_unchanged/is_received_locally into TransferMetadata (#37341) Signed-off-by: Sage Moore <sage@neuralmagic.com> * [ROCm] Hotfix: guard MLA dual RMS norm fusion against older AITer versions (#40386) Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com> * [Misc][UX] Suppress confusing `num_gpu_blocks` log lines (#40402) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix] Replace code that disabled shared expert overlap (#39222) Signed-off-by: Bill Nell <bnell@redhat.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> * [vLLM IR] Add IR op testing and benchmarking infrastructure (#40167) Signed-off-by: Yanan Cao <gmagogsfm@gmail.com> Co-authored-by: Theresa Shan <Theresa.Shan@amd.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [Bugfix] Fix `_CONFIG_REGISTRY` types getting wrong config class when on-disk model_type differs (#39554) Signed-off-by: Misa <misaAle@users.noreply.github.com> Signed-off-by: Misael Casarez <misacasa@amazon.com> Co-authored-by: Misael Casarez <misacasa@amazon.com> * [Misc] Reduce attention logging levels (#40086) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com> * [Bugfix] Gemma4: fix multimodal embedder norm order to match HF reference (#40411) Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com> Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com> * [Doc] Update ViT CUDA graph doc for mixed (image+video) inputs (#40355) Signed-off-by: shen-shanshan <467638484@qq.com> * [Startup] Parallelize torch/transformers import + weight prefetch + forkserver prewarm (#40331) Signed-off-by: simon-mo <simon@inferact.ai> * [Deprecation] Deprecate cprofile and cprofile_context (#39100) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Fix] Add missing space in IP fallback warning (#40359) Signed-off-by: lesj0610 <lesj0610@gmail.com> * [MM][Misc] Support image+video mixed inputs (per prompt) for VLM examples (#40335) Signed-off-by: shen-shanshan <467638484@qq.com> * [MoE] Triton MoE Perf regression - restore low latency path (#39016) * [Feat] dflash support for ROCm (#39703) Signed-off-by: Hang Yang <hangy@amd.com> * [Bugfix] Normalize malformed dict prompts that carry token IDs in `prompt` (#40339) Signed-off-by: Alchuang22-dev <2584829494@qq.com> * Revert "[Startup] Parallelize torch/transformers import + weight prefetch + forkserver prewarm" (#40438) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> * fix: clamp NaN/Inf in topk_softmax to prevent duplicate expert IDs (#39391) Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com> * feat(multimodal): support externally processed mm_kwargs with cache injection (#39502) Signed-off-by: Krish Hung <krishung5@gmail.com> Signed-off-by: krishung5 <krish@nvidia.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [Frontend] Remove frontend pooling multi task support. (#37861) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * Add Granite 4.1 Vision as built-in multimodal model (#40282) Signed-off-by: Artem Spector <artems@il.ibm.com> Signed-off-by: artemspector <artems@il.ibm.com> Co-authored-by: artemspector <artems@il.ibm.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * [Bugfix] Fix dataset name and path argument validation bug in vllm bench serve (#40288) Signed-off-by: talora <talora@nvidia.com> Signed-off-by: Talor Abramovich <talor19@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [Doc] Add Qwen3 AWQ models to documentation (#40034) Signed-off-by: Yusuf <yusufmohammad@live.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix] LoRA: extend expert base_layer loading to Qwen3.5 and Step3.x (#37114) Signed-off-by: Hollow Man <hollowman@opensuse.org> * [XPU][CI] Add misc, engine and lora cases on Intel GPU in CI (#39887) Signed-off-by: zengxian <xiangdong.zeng@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [MM][CG] Optimize default `max_frames_per_batch` auto-infer for ViT CUDA graph video inference (#40445) Signed-off-by: shen-shanshan <467638484@qq.com> * Default to 'align' mamba cache mode for Mamba-based models when speculative decoding is enabled (#40454) Signed-off-by: Roi Koren <roik@nvidia.com> * [UX] Bump version in CG memory profiling log message (#40465) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> * Revert #38730 and #38791 (#40032) Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com> Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com> * [Model Runner V2] Multiple prompt logprobs support (#39937) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> * Add new tp plan styles to the Transformers modelling backend (#40467) Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> * Revert "[Misc] Move `pyav` and `soundfile` to common requirements" (#40276) Co-authored-by: Roger Wang <hey@rogerw.io> * [MRv2]fix: model accuracy regression caused by reusing the stale last_sampled_tokens and draft_tokens (#39833) Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> * [Bugfix] Fix spec decode test failures on Blackwell (SM100+) (#39546) Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com> Signed-off-by: Rishi Puri <puririshi98@berkeley.edu> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Stefano Castagnetta <scastagnetta@nvidia.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Benjamin Chislett <bchislett@nvidia.com> * [Performance] Add is_reasoning_end_streaming() override to GptOssReasoningParser (#35745) Signed-off-by: Fergus <fergus.barratt00@gmail.com> Signed-off-by: fergus barratt <fergus.barratt00@gmail.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com> * [Bugfix][Kernel] nvfp4 cutlass MoE: fix nvfp4 experts quant out-of-bounds read for expert counts not divisible by 4 or 16 (#40351) Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com> * [Perf] Optimize batch invariant with fused rms norm, 2.1% E2E latency improvement (#40413) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Refactor] Remove unused param (#39750) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [MoE Refactor] Remove SharedFusedMoE class (#35782) Signed-off-by: Bill Nell <bnell@redhat.com> * [MoE Refactor] Add more MoE layer tests (#39349) Signed-off-by: Bill Nell <bnell@redhat.com> Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> * [Startup][UX] Enable CUDAGraph memory profiling by default (#38284) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com> * [ROCm] [Wheel] [Bugfix] [Critical] Remove any packages installed from github from rocm.txt e.g `fastsafetensors` as it is incompatible with `uv pip` (#40461) Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com> * [Bugfix] avoid warmup if text only expectation in multi_modal run (#40409) Signed-off-by: khushali9 <khushali.desai9@gmail.com> * test: add nan/inf clamp regression test for fused_topk_bias (#40553) Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com> * [ROCm][P/D][MORI][BugFix] Ensure correct api is used when making requests to prefill / decode nodes (#39835) Signed-off-by: Randall Smith <Randall.Smith@amd.com> * [NIXL][XPU]Fix nixl import on XPU (#40430) Signed-off-by: Soila Kavulya <soila.p.kavulya.intel.com> * [Bugfix] Fix W4A8_FP8 MoE tp>1 correctness and view() TypeError (#40310) Signed-off-by: EdalatiAli <aliedalati@cohere.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [AMD][CI][BugFix] Override normalize_e4m3fn_to_e4m3fnuz for fnuz machines in test_moe_layer_no_parallel (#40550) Signed-off-by: Randall Smith <Randall.Smith@amd.com> * [Multimodal] Add PyAV video backend for concurrent video decoding (#39986) Signed-off-by: Jaseel Muhammad <jaseel.muhammad@mbzuai.ac.ae> Signed-off-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn> * [MyPy] Enable mypy for `vllm/model_executor/layers/` (#40159) Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com> * [Bugfix] Fix quantized model initialization failure with prefetch offloading (#40432) Signed-off-by: Rishapveer Singh <singhrishapveer@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [compile] mla + group fp8 fusion (#38877) Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [ROCm] Support non-causal attention in ROCM_ATTN (#40176) Signed-off-by: Micah Williamson <micah.williamson@amd.com> * [Audio] Bundle `get_generation_prompt()` params into `SpeechToTextParams` (#36268) Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> * [Bugfix] Pass effective chat template kwargs to reasoning parsers (#40460) Signed-off-by: Bugen Zhao <i@bugenzhao.com> * [kv_offload+HMA][8/N]: Support multi-group worker transfer (#38453) Signed-off-by: Or Ozeri <oro@il.ibm.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [fix] flaky test_mla_attn_quant_fusion.py (#40530) Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com> * [Model] Add block-local attention and YaRN for local layers to Gemma3 (#39823) Signed-off-by: Philip Monk <169196560+philip-essential@users.noreply.github.com> * [Bugfix] [Reasoning] Add reasoning_start_str/reasoning_end_str properties to reasoning parsers (#40566) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com> * [Multimodal] Support custom video metadata for pre-extracted frame sequences (#40133) Signed-off-by: storyicon <storyicon@foxmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * upgrade tpu-inference to v0.18.0 (#40395) * support hotwords for FunASR model (#39674) Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com> Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com> * [Bugfix][CPU][RISC-V] Clamp exp() input to prevent NaN (#40428) Signed-off-by: liuyudong <liuyudong@iscas.ac.cn> * [Bugfix] LoRA for DeepSeek V3.2 (#35077) Signed-off-by: Hollow Man <hollowman@opensuse.org> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> * [Doc] Clarify supported keys for --speculative-config (#40455) Signed-off-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com> Co-authored-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com> * [xpu][rocm] Update `current_platform.supports_fp8()` for TritonExperts (#40132) Signed-off-by: Hank <hcc.mayday@gmail.com> * [Bugfix][CI] Fix `v1/kv_connector/unit/test_nixl_connector_hma.py::test_fewer_blocks_with_hma` (#40597) Signed-off-by: NickLucche <nlucches@redhat.com> * [MoE Refactor] Combine MoERunnerBase + DefaultMoERunner (#40560) Signed-off-by: Bill Nell <bnell@redhat.com> * [MoE] Convert CT W8A8 To Oracle Structure (#39187) Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> * [NVFP4] NVFP4 MOE emulation fallback for H100/MI300/MI350, standardize `TritonExperts` usage for OCP MX emulation (#35737) Signed-off-by: Felix Marty <Felix.Marty@amd.com> Signed-off-by: fxmarty-amd <felmarty@amd.com> Co-authored-by: Kyle Sayers <kylesayrs@gmail.com> * Fix test_startup.py for torch 2.12 (#40636) Signed-off-by: Angela Yi <yiangela7@gmail.com> * FlexAttention non-causal support (#40394) Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix][Parser] Fix Mistral pre-v11 tool parser failing on trailing model output (#40531) Signed-off-by: dougbtv <dosmith@redhat.com> Signed-off-by: Doug Smith <dougbtv@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> * [Refactor] Clean up log once `scope="local"` (#40540) Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [Bugfix][Torch 2.12] Fix batch_invariant test with allow_override for torch 2.12 upgrade (#40562) Signed-off-by: Lucas Kabela <lucaskabela@meta.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [gRPC] Add standard gRPC health checking (grpc.health.v1) for Kubernetes native probes (#38016) Signed-off-by: Honglin Cao <Caohonglin317@hotmail.com> * [Bugfix] Fix RMS norm + quant fusion on DeepGEMM UE8M0 path for B200 (#40552) Signed-off-by: Lucas Kabela <lucaskabela@meta.com> * [Fix][MoRI] Align MoRI-IO message format with P2pNcclConnector and vllm-router (#39565) Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com> Co-authored-by: Matvei Pashkovskii <mpashkov@amd.com> * [BE] Fix compile time message to be consistent (use monitoring) (#40641) Signed-off-by: Lucas Kabela <lucaskabela@meta.com> * [XPU] disable fusion pattern support on XPU platform (#39789) Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [TurboQuant] enable FA3/FA4 for prefill paths (#40092) Signed-off-by: 墨楼 <huangzhilin.hzl@antgroup.com> Co-authored-by: 墨楼 <huangzhilin.hzl@antgroup.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Codex <codex@openai.com> * [BugFix]fix Qwen3 MoE call gate twice (#40664) Signed-off-by: Kunshang Ji <kunshang.ji@intel.com> * [MoE refactor] refactor GPTQMarlinMoEMethod with MK (#37990) Signed-off-by: Kunshang Ji <kunshang.ji@intel.com> Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: Robert Shaw <robertgshaw2@gmail.com> * [compile] Skip FX graph deserialiaztion on loading, further reducing warm compile time. (#40151) Signed-off-by: zhxchen17 <zhxchen17@fb.com> * [Responses] Add tool_choice/tools validation to match OpenAI behavior (#40399) Signed-off-by: sfeng33 <4florafeng@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [MM][CG] Support `--enable-vit-cuda-graph` option for VLM examples (#40580) Signed-off-by: shen-shanshan <467638484@qq.com> * [Misc] Support Human-readable (k/K/m/M..) json cli arg (#40473) Signed-off-by: NickLucche <nlucches@redhat.com> * [Build] Switch default CUDA to 13.0, update CUDA architecture lists, clean up stale build-args (#39878) Signed-off-by: Shengqi Chen <harry-chen@outlook.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * [UT][Hardware] let torchrun example tests use the default backend (#39879) Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com> * [ROCm] Add gfx1102/gfx1103 support (#40037) Signed-off-by: Matthias Gehre <matthias.gehre@amd.com> * [Compilation] Refactor SiluMul activation+quant Fusion Pass (#39684) Signed-off-by: BadrBasowid <badr.basowid@gmail.com> * [CPU] Added faster exp routine for lower precision data types. (#38112) Signed-off-by: Anna Mayne <anna.mayne@arm.com> Co-authored-by: Fadi Arafeh <fadi.arafeh@arm.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> * [XPU][CI]Temporary disable 3 cases on Intel GPU in CI (#40683) Signed-off-by: zengxian <xiangdong.zeng@intel.com> * [Misc] use model arch converter for bidi models identification (#40701) Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> * [Bugfix] Include inductor and functorch configs in compilation cache key (#40627) Signed-off-by: Richard Zou <zou3519@gmail.com> * [Model] Support Hy3 preview (#40681) Signed-off-by: stevenkuang <stevenkuang@tencent.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> * [Bugfix] Make Attention Backend Auto-Selection Batch-Invariance-Aware (#40193) Signed-off-by: Srreyansh Sethi <srreyansh.sethi@gmail.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [XPU] Upgrade torch 2.11 for xpu (#37947) Signed-off-by: Kunshang Ji <kunshang.ji@intel.com> * [ROCm] Implement GPU-to-NUMA-node detection (#40015) Signed-off-by: Patrick Schlangen <pschlan@amd.com> Co-authored-by: TJian <tunjian.tan@embeddedllm.com> * [CI] Split disaggregated tests into own test-area (#40623) Signed-off-by: NickLucche <nlucches@redhat.com> * [MoE Refactor] Rename FusedMoE.make_expert_params_mapping to fused_moe_make_expert_params_mapping (#40671) Signed-off-by: Bill Nell <bnell@redhat.com> * [kv_offload+HMA][10/N]: Support load with multiple KV groups (#39402) Signed-off-by: Or Ozeri <oro@il.ibm.com> * [DP][Ray] Pin DP control bundle to same node as first GPU bundle (#39167) Signed-off-by: Shahar Mor <smor@nvidia.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [MoE] Move xpu moe to fused_moe/experts/ (#40568) Signed-off-by: Jackmin801 <ongjackm@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [NVIDIA] Add sm_110 (Jetson Thor) to CUDA 13.0 build targets (#39233) * [Bugfix] Fix workspace resize leaking reserved GPU memory (#39226) Signed-off-by: root <conway.zhu@cohere.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Bugfix] Fix DeepSeek V2-Lite Accuracy drop (#40673) Signed-off-by: Bill Nell <bnell@redhat.com> * [Spec Decode] Move `SpecDecodeBaseProposer` out of `eagle.py` (#40732) Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> * [Docs]Add documentation for bench serve visualization arguments (#40539) Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [MoE] Move remaining PrepareAndFinalize to prepare finalize folder (#39009) Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Signed-off-by: Jackmin801 <ongjackm@gmail.com> Co-authored-by: Robert Shaw <robertgshaw2@gmail.com> * [ROCm] Cast score correction bias tensor during model construction for DeepSeek/Kimi-K2 (#39999) Signed-off-by: Hemanth Acharya <heachary@amd.com> * [EPLB] Remove asyncio infrastructure from Async EPLB (#40730) Signed-off-by: Sage Moore <sage@neuralmagic.com> * [Core] Avoid seq_lens_cpu GPU->CPU sync (#40654) Signed-off-by: Nick Hill <nickhill123@gmail.com> * [Feat] Unified Synthetic Acceptance Rate for V1 and V2 (#40662) Signed-off-by: Benjamin Chislett <bchislett@nvidia.com> Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [Bugfix] Treat <tool_call> as implicit reasoning end in Qwen3 parser (#35687) Signed-off-by: Doug Campos <qmx@qmx.me> * [Misc] Added curl retries in install_python_libraries.sh (#36700) Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [BUG]: fix HF tokenizer concurrent borrow in tool parsers (#40059) Signed-off-by: Yifan <yzong@redhat.com> Co-authored-by: timon0305 <timon0305@outlook.com> Co-authored-by: sfeng33 <4florafeng@gmail.com> * [MRV2] Ensure warmup covers prefill path (#40746) Signed-off-by: Nick Hill <nickhill123@gmail.com> * [Doc] fix capitalization consistency in README (vLLM, Hugging Face) (#40729) Signed-off-by: Vinayak Mishra <vinayakmishra448@gmail.com> * [Feature] Avoid eager import of the "mistral_common" package. (#40043) Signed-off-by: Neil Schemenauer <nas@arctrix.com> * [Platform] Fix RISC-V platform detection (lscpu parsing + non-NUMA meminfo) (#40427) Signed-off-by: liuyudong <liuyudong@iscas.ac.cn> * [Bugfix] Avoid mutating `chat_template_kwargs` in `HYV3ReasoningParser` initialization (#40713) Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * [Deprecate] Deprecate LLM.reward offline api, use LLM.encode instead. (#40688) Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: wang.yuqi <noooop@126.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> * [MoE] Move cutlass moe to fused_moe/experts/ (#40574) Signed-off-by: Jackmin801 <ongjackm@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> * [XPU][CI] Fix Docker cleanup races on Intel CI runners (#40761) Signed-off-by: zengxian <xiangdong.zeng@intel.com> * Support only half types for concat_mla_q kernel (#37892) Signed-off-by: Xin Yang <xyangx@amazon.com> * [XPU] Enable torch.compile for XPU GDN attention (#39466) Signed-off-by: yuwenzho <yuwen.zhou@intel.com> Signed-off-by: Yuwen Zhou <yuwen.zhou@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> * [Model] Gemma4: add bidirectional vision attention for sliding layers with window guard (#40534) Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com> Signed-off-by: Luciano Martins <lucianomartins@google.com> Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com> Co-authored-by: Isotr0py <2037008807@qq.com> Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn> * [Bugfix] Fix IMA in DSA + MTP (#40772) Signed-off-by: Woosuk Kwon <woosuk@inferact.ai> * Fix Nano Nemotron VL static image inputs (#40724) Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> * [CI/Build] Add e2e test for ViT CUDA graph (#40780) Signed-off-by: shen-shanshan <467638484@qq.com> * [Build] Bump CUDA to 13.0.2 to match PyTorch 2.11.0 (#40669) Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com> * [Bugfix][Parser] Fix Mistral tool parser for HF tokenizers (#39294) Signed-off-by: thomasmaindron <thomasmaindron@users.noreply.github.com> Co-authored-by: thomasmaindron <thomasmaindron@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com> * [Attention] use diff kv backend for mimo v2 flash (#40045) Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> * [Bugfix][MoE] Unpad routed output before shared expert add [Fixes #35949] (#40794) Signed-off-by: Netanel Haber <nhaber@nvidia.com> * [kv_offload+HMA][9/N]: Support lookup with multiple KV groups (#39401) Signed-off-by: Or Ozeri <oro@il.ibm.com> * fused_moe: treat NIXL EP as batched experts (#40412) Signed-off-by: Itay Alroy <ialroy@nvidia.com> * [Quantization] add humming quantization kernel (#34556) * [Refactor] Unify 2D/3D kernels in triton_unified_attention (#40631) Signed-off-by: JartX <sagformas@epdcenter.es> * [Docs] Add docs for context extension using the yarn method (#37430) Signed-off-by: xiaoming <1259730330@qq.com> Signed-off-by: labAxiaoming <34019940+labAxiaoming@users.noreply.github.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> * [Frontend] Delegate to vLLM Omni When `--omni` Passed (#40744) Signed-off-by: Alex Brooks <albrooks@redhat.com> * [Build] Add Python 3.14 to supported version list. (#34770) Signed-off-by: Neil Schemenauer <nas@arctrix.com> Co-authored-by: Simon Mo <simon.mo@hey.com> * [Bug] Fix GLM-5.1 running error on ROCm platform (#40763) Signed-off-by: Qiang Li <qiang.li2@amd.com> * [Bugfix][CI] Fix wrong residual shape in TestFusedAddRMSNorm.example_inputs that causes flaky test (#40629) Signed-off-by: Zhang Jian <jianmusings@gmail.com> * [EPLB] Fix replica selection bias in fused_moe router (#40810) Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com> * [Bugfix] add seq_lens_cpu_upper_bound to CommonAttentionMetadata in mla_runner.py (#40844) Signed-off-by: ignaciosica <mignacio.sica@gmail.com> * [BE][Bugfix] Respect TORCH_COMPILE_DISABLE env var at the vLLM config level for torch 2.12 (#40715) Signed-off-by: Lucas Kabela <lucaskabela@meta.com> * [Refactor] Remove unused dead code (#40640) Signed-off-by: yewentao256 <zhyanwentao@126.com> * [Test] Increase qwen2_vl num_logprobs to fix torch 2.12 update (#40818) Signed-off-by: Angela Yi <angelayi@meta.com> * [ROCm][CI] Fix TestSiluMulGroupFp8QuantModel after W8A8 block linear refactor (#39799) Signed-off-by: Andreas Karatzas <akaratza@amd.com> * [ROCm][CI] Fix `trust_remote_code` AttributeError in EAGLE3 acceptance length test (#40306) Signed-off-by: Andreas Karatzas <akaratza@amd.com> * [Feature] Warm up readonly multimodal processor during renderer startup (#40797) Signed-off-by: Chenguang ZHENG <645327136@qq.com> Co-authored-by: OpenAI Codex <codex@openai.com> * [ROCm][Engine] Fix GPU memory leaks in engine shutdown and test workaround for async KV prefix cache reset (#38503) Signed-off-by: Andreas Karatzas <akaratza@amd.com> * [Opt] Optimize deepstack buffer handling for multimodal Qwen3 models (#40145) Signed-off-by: xiaoming <1259730330@qq.com> * [CI][AMD]BugFix] Fix deadlock occuring in test_moe_layer (#40767) Signed-off-by: Randall Smith <Randall.Smith@amd.com> * [kv_offload+HMA][11/N]: Support store with multiple KV groups (#39403) Signed-off-by: Or Ozeri <oro@il.ibm.com> * [Bugfix][MoE] Only unpad routed output before shared expert add or routed output transform (#40865) Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> * [Bugfix] Fix the DSML token leakage in DSV4/3.2 (#40806) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com> Signed-off-by: sfeng33 <4florafeng@gmail.com> Co-authored-by: sfeng33 <4florafeng@gmail.com> Co-authored-by: Windswithyou 1694599440@qq.com * [LoRA] MoE LoRA Refactor (#40338) * [Bugfix] Size FlashInfer NVLink MNNVL workspace to EP group (#40893) Signed-off-by: Dao Le <Dao007forever@gmail.com> * [KV Offload] Offload all KV blocks when doing prefill in P/D (#40346) Signed-off-by: omerpaz95 <omerpaz95@gmail.com> Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com> Co-authored-by: Or Ozeri <or@ozery.com> * [torch.compile]: Disable Sequence Parallelism (SP) for piecewise compilation (#38373) Signed-off-by: SouthWest7 <am1ao@qq.com> Signed-off-by: Xinan Miao <1403572259@qq.com> Co-authored-by: SouthWest7 <am1ao@qq.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: OpenAI Codex <codex@openai.com> Co-authored-by: Wang Xingran <72983099+wangxingran222@users.noreply.github.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> * [Feat] DeepSeek V4 Rebased (#40860) Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Signed-off-by: Woosuk Kwon <woosuk@inferact.ai> Signed-off-by: qizixi <zixi@inferact.ai> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Yongye Zhu <yongye@inferact.ai> Co-authored-by: Simon Mo <simon@inferact.ai> Co-authored-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Roger Wang <hey@rogerw.io> Co-authored-by: Roy Wang <yasong.wang@inferact.ai> Co-authored-by: Woosuk Kwon <woosuk@inferact.ai> Co-authored-by: youkaichao <youkaichao@gmail.com> Co-authored-by: Zhewen Li <jerven.vllm@gmail.com> Co-authored-by: Zijing Liu <liuzijing2014@gmail.com> Co-authored-by: khluu <khluu000@gmail.com> Co-authored-by: qizixi <zixi@inferact.ai> Co-authored-by: Zhewen Li <zhewenli@inferact.ai> * Revert "[Frontend] Remove frontend pooling multi task support. (#37861)" This reverts commit d2e2e856ad4daf4b36bb37acaac06f3f61ac84ce. * [Bugfix] Cap SWA/chunked-local runtime admission to startup pool-sizing bound (#40946) Signed-off-by: Dao Le <Dao007forever@gmail.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> (cherry picked from commit 7b1bc0a3eb01a6bc2650eda9970049f7825240d7) * [DSV4] Add silu clamp limit to shared expert (#40950) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> (cherry picked from commit 706a04d34ba64ea23d430d5e50038791aacfae96) * [Docker] Install numactl CLI in CUDA runtime image (#41032) Signed-off-by: Zhewen Li <zhewenli@inferact.ai> Co-authored-by: Zhewen Li <zhewenli@inferact.ai> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> (cherry picked from commit 6232fb4b66b42c8e5f4ef1cc4c5163442cc99208) * Temporary disable persistent topk (#41442) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> * Auto-disable expandable_segments around cumem memory pool (#40812) Signed-off-by: youkaichao <youkaichao@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> (cherry picked from commit 2ce95a761b9acd925d1bc69cfd1d4fc13de9e2b7) * [Model][DSV4] Support base model (#41006) Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> (cherry picked from commit 2c8b76c5cb2683f05650f20d90a63f3d9799e909) * [DSV4] Enable Multi-stream for Pre-Attn GEMM (#41061) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> (cherry picked from commit 5aa371dc8e38e053754d89b444abca0a1d63f676) * [Core] Account for `num_gpu_blocks_override` in `max_model_len` checks (#41069) Signed-off-by: Nick Hill <nickhill123@gmail.com> (cherry picked from commit e68fa1b90a7bc52510c11fe2edeae11db15f98fc) * [DSV4] Align aux stream API with DeepseekV4DecoderLayer (#41171) Signed-off-by: zixi-qi <zixi@inferact.ai> (cherry picked from commit 6fb3f7b46b12ea63265afbe6d53d6f15a5de7b3a) * [Bugfix] Fix Deepseek V4 import error due to AOT compile cache loading (#41090) Signed-off-by: wzhao18 <wzhao18.sz@gmail.com> Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com> (cherry picked from commit 803b9d7881cd3a8482aaa1e6bf990193b55c6331) * [Bugfix] fix inductor error for dpsk v4 (#41135) Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> (cherry picked from commit 2ae73c758ceed55ad2f70a69b47c8a994fce5662) * [Bugfix] Fix max_num_batched_token not captured in cuda graph (#40734) Signed-off-by: wzhao18 <wzhao18.sz@gmail.com> Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com> Co-authored-by: Wei Zhao (Engrg-Hardware 1) <weizha@login-bia02.bia.clusters.nvidia.com> (cherry picked from commit 8b49cf3a37eb1a267a06b0df23328909330af1e6) * [Bugfix] DSV32/V4 add missing type conversion for non-streaming tool calls (#41198) Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com> (cherry picked from commit 762022cafb1afc4c51ce706c043e2f1f5826003a) * [Bugfix] Fix repeated DSv4 RoPE cache initialization (#41148) Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> (cherry picked from commit 9d8ad5b408bf447e41a3629fc21a453720aaf52b) * [DSv4] Use `cvt` PTX for FP32->FP4 conversion (#41015) Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> (cherry picked from commit 296741d0257107a9d0301409005c85d38bb247bc) * [Bugfix] Fix persistent_topk cooperative deadlock at TopK=1024 (#41189) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> (cherry picked from commit a749a33d8d05acdd3ab346bd3f0c6b5c9c80474f) * [DSV4] Add BF16 and MXFP8 A2A support for flashinfer a2a one sided (#40960) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> Co-authored-by: Zijing Liu <liuzijing2014@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> (cherry picked from commit b4806c8ee12d5c5bbfebd6070b389e2f4daad1fd) * . Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> (cherry picked from commit db0c9d52d44210937e2a9e10317ab3e06e40c53a) Signed-off-by: khluu <khluu000@gmail.com> * [Bugfix ] fix bailing_moe_linear (#40859) Signed-off-by: ghphotoframe <854746559@qq.com> (cherry picked from commit ed57f771923703998a17ad656536ffb460447a2c) * [Bugfix] BailingMoeV2.5: rotate full qk_rope_head_dim in MLA RoPE (#41185) Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> (cherry picked from commit 6d7d4da99e41c4ccc0d52d74e2bf36d1ff31034d) * [Bugfix] Pass reasoning parser kwargs to structured output (#41199) Signed-off-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io> (cherry picked from commit a07642667db1284ad2128d7f9ef089e6b0d24a4c) * [Perf] Intergrate Tile Kernels `head_compute_mix_kernel` for Deepseek-V4 (#41255) Signed-off-by: Isotr0py <Isotr0py@outlook.com> Co-authored-by: Roger Wang <hey@rogerw.io> (cherry picked from commit a9484dac7b734096ed26db4902454da7e497d2c3) * [ROCm][Bugfix][GPTOSS]: fix input_ids and expert_map args for quark w4a8 gptoss (#41165) Signed-off-by: Rohan138 <rohanpotdar138@gmail.com> (cherry picked from commit 3795d7acf431980e62e738493f437ae2a51549da) * Revert "." This reverts commit a5e0ea09ea7ad66f90322f1da308152574867779. * [DSV4] Add knob to enable pre-attn gemm (#41443) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> (cherry picked from commit bcf5cac9fb956788f649d1f5297b74c886a9d6d3) * Build Python from source instead of using deadsnakes PPA Replace the slow/flaky add-apt-repository deadsnakes PPA with building Python from source. This avoids the long wait times and retry loops when Launchpad PPA servers are slow or unresponsive. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: khluu <khluu000@gmail.com> * [Bugfix] Fix persistent_topk inter-CTA init race on RadixRowState (#41444) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> (cherry picked from commit edd60ac93a3247c7ef1bf1e2a3e9c0e95bc83bf6) * Revert "Temporary disable persistent topk (#41442)" This reverts commit 05ebca525009a4fe3dc89ff53de6469cb2ac0800. * [DSV4] Guard megamoe flag with Pure TP (#41522) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> * [DSv4] Tune default value of `VLLM_MULTI_STREAM_GEMM_TOKEN_THRESHOLD` (#41526) Co-authored-by: Copilot <copilot@github.com> * Temporary disable persistent topk for Hopper (#41605) Signed-off-by: zixi-qi <zixi@inferact.ai> * Revert "[DSv4] Use `cvt` PTX for FP32->FP4 conversion (#41015)" This reverts commit d9f3961abbe66333a249fc5eb3cd832766dee02c. * feat: task1 ~ task3 추가 * feat: task4 ~ task6 추가 * feat: task7 ~ task8 추가 * feat: task9 추가 * feat: add QuotaServe soft-quota cache eviction to vLLM * feat: add extensible KV cache eviction metrics * Tune QuotaServe reuse weight to 0.2 Assisted-by: OpenAI Codex * Set QuotaServe reuse weight to zero Assisted-by: OpenAI Codex * Align QuotaServe quotas with compute token demand Assisted-by: Codex * feat: compute QuotaServe quotas from observed KV footprints Signed-off-by: zero-uuuuk <bike481227@gmail.com> * feat: retain session demand across idle gaps and reclaim expired caches Signed-off-by: zero-uuuuk <bike481227@gmail.com> * fix: return an error when prefix cache reset fails Signed-off-by: zero-uuuuk <bike481227@gmail.com> * feat: add FIFO and LFU prefix cache eviction policies Signed-off-by: zero-uuuuk <bike481227@gmail.com> * docs: describe footprint quotas and block eviction accounting Signed-off-by: zero-uuuuk <bike481227@gmail.com> * Protect recently released prefixes during quota reclaim Signed-off-by: zero-uuuuk <bike481227@gmail.com> * Account arrived prompt demand before request completion Signed-off-by: zero-uuuuk <bike481227@gmail.com> * Retire superseded session contexts before live cache eviction Signed-off-by: zero-uuuuk <bike481227@gmail.com> * Allow observed return-gap jitter during quota reclaim Signed-off-by: zero-uuuuk <bike481227@gmail.com> * Document final session-aware GPU eviction policy Signed-off-by: zero-uuuuk <bike481227@gmail.com> --------- Signed-off-by: Yuan Tang <terrytangyuan@gmail.com> Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com> Signed-off-by: chaojun-zhang <chaojun.zhang@intel.com> Signed-off-by: Julien Denize <julien.denize@mistral.ai> Signed-off-by: juliendenize <julien.denize@mistral.ai> Signed-off-by: velonica0 <like@mail.nankai.edu.cn> Signed-off-by: San-Nguyen <san.nguyen@ibm.com> Signed-off-by: wuyingjun <wuyingjun_yewu@cmss.chinamobile.com> Signed-off-by: milesial <milesial@users.noreply.github.com> Signed-off-by: Andreas Karatzas <akaratza@amd.com> Signed-off-by: nemanjaudovic <nudovic@amd.com> Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com> Signed-off-by: Or Ozeri <oro@il.ibm.com> Signed-off-by: ilmarkov <markovilya197@gmail.com> Signed-off-by: Markov Ilya <markovilya19@gmail.com> Signed-off-by: Zhanqiu Hu <zhu@redhat.com> Signed-off-by: Yan Ma <yan.ma@intel.com> Signed-off-by: Aleksandar Yanakiev <alexander.yanakiev@discretestack.com> Signed-off-by: kse <kevin.sejourne@cloud-temple.com> Signed-off-by: Vasiliy Kuznetsov <vasiliy@meta.com> Signed-off-by: Sage Moore <sage@neuralmagic.com> Signed-off-by: larryli2-amd <larryli2@amd.com> Signed-off-by: yewentao256 <zhyanwentao@126.com> Signed-off-by: Rita Brugarolas Brufau <rita.brugarolasbrufau@amd.com> Signed-off-by: Hashem Hashemi <hashem.hashemi@amd.com> Signed-off-by: Bill Nell <bnell@redhat.com> Signed-off-by: Frederik Gossen <frgossen@meta.com> Signed-off-by: bai <v@gor.io> Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com> Signed-off-by: NickLucche <nlucches@redhat.com> Signed-off-by: Theresa Shan <thshan@smci355-ccs-aus-n08-21.prov.aus.ccs.cpe.ice.amd.com> Signed-off-by: Theresa Shan <theresa.shan@amd.com> Signed-off-by: aeon-x <talexcao@gmail.com> Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: Yanan Cao <gmagogsfm@gmail.com> Signed-off-by: Misa <misaAle@users.noreply.github.com> Signed-off-by: Misael Casarez <misacasa@amazon.com> Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com> Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com> Signed-off-by: shen-shanshan <467638484@qq.com> Signed-off-by: simon-mo <simon@inferact.ai> Signed-off-by: lesj0610 <lesj0610@gmail.com> Signed-off-by: Hang Yang <hangy@amd.com> Signed-off-by: Alchuang22-dev <2584829494@qq.com> Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io> Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com> Signed-off-by: Krish Hung <krishung5@gmail.com> Signed-off-by: krishung5 <krish@nvidia.com> Signed-off-by: wang.yuqi <noooop@126.com> Signed-off-by: Artem Spector <artems@il.ibm.com> Signed-off-by: artemspector <artems@il.ibm.com> Signed-off-by: talora <talora@nvidia.com> Signed-off-by: Talor Abramovich <talor19@gmail.com> Signed-off-by: Yusuf <yusufmohammad@live.com> Signed-off-by: Hollow Man <hollowman@opensuse.org> Signed-off-by: zengxian <xiangdong.zeng@intel.com> Signed-off-by: Roi Koren <roik@nvidia.com> Signed-off-by: Vadim Gimpelson <vadim.gimpelson@gmail.com> Signed-off-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com> Signed-off-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Signed-off-by: Zijing Liu <liuzijing2014@gmail.com> Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com> Signed-off-by: Rishi Puri <puririshi98@berkeley.edu> Signed-off-by: Fergus <fergus.barratt00@gmail.com> Signed-off-by: fergus barratt <fergus.barratt00@gmail.com> Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com> Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com> Signed-off-by: khushali9 <khushali.desai9@gmail.com> Signed-off-by: Randall Smith <Randall.Smith@amd.com> Signed-off-by: Soila Kavulya <soila.p.kavulya.intel.com> Signed-off-by: EdalatiAli <aliedalati@cohere.com> Signed-off-by: Jaseel Muhammad <jaseel.muhammad@mbzuai.ac.ae> Signed-off-by: Isotr0py <2037008807@qq.com> Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com> Signed-off-by: Rishapveer Singh <singhrishapveer@gmail.com> Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com> Signed-off-by: Micah Williamson <micah.williamson@amd.com> Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com> Signed-off-by: Philip Monk <169196560+philip-essential@users.noreply.github.com> Signed-off-by: storyicon <storyicon@foxmail.com> Signed-off-by: zixiao <shunli.dsl@alibaba-inc.com> Signed-off-by: liuyudong <liuyudong@iscas.ac.cn> Signed-off-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com> Signed-off-by: Hank <hcc.mayday@gmail.com> Signed-off-by: Robert Shaw <robertgshaw2@gmail.com> Signed-off-by: Felix Marty <Felix.Marty@amd.com> Signed-off-by: fxmarty-amd <felmarty@amd.com> Signed-off-by: Angela Yi <yiangela7@gmail.com> Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com> Signed-off-by: dougbtv <dosmith@redhat.com> Signed-off-by: Doug Smith <dougbtv@gmail.com> Signed-off-by: Lucas Kabela <lucaskabela@meta.com> Signed-off-by: Honglin Cao <Caohonglin317@hotmail.com> Signed-off-by: simondanielsson <simon.danielsson99@hotmail.com> Signed-off-by: Chaojun Zhang <chaojun.zhang@intel.com> Signed-off-by: 墨楼 <huangzhilin.hzl@antgroup.com> Signed-off-by: Kunshang Ji <kunshang.ji@intel.com> Signed-off-by: zhxchen17 <zhxchen17@fb.com> Signed-off-by: sfeng33 <4florafeng@gmail.com> Signed-off-by: Shengqi Chen <harry-chen@outlook.com> Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com> Signed-off-by: Matthias Gehre <matthias.gehre@amd.com> Signed-off-by: BadrBasowid <badr.basowid@gmail.com> Signed-off-by: Anna Mayne <anna.mayne@arm.com> Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn> Signed-off-by: Richard Zou <zou3519@gmail.com> Signed-off-by: stevenkuang <stevenkuang@tencent.com> Signed-off-by: Srreyansh Sethi <srreyansh.sethi@gmail.com> Signed-off-by: Patrick Schlangen <pschlan@amd.com> Signed-off-by: Shahar Mor <smor@nvidia.com> Signed-off-by: Jackmin801 <ongjackm@gmail.com> Signed-off-by: root <conway.zhu@cohere.com> Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com> Signed-off-by: Hemanth Acharya <heachary@amd.com> Signed-off-by: Nick Hill <nickhill123@gmail.com> Signed-off-by: Benjamin Chislett <bchislett@nvidia.com> Signed-off-by: Benjamin Chislett <chislett.ben@gmail.com> Signed-off-by: Doug Campos <qmx@qmx.me> Signed-off-by: Dmitry Tokarev <dtokarev@nvidia.com> Signed-off-by: Yifan <yzong@redhat.com> Signed-off-by: Vinayak Mishra <vinayakmishra448@gmail.com> Signed-off-by: Neil Schemenauer <nas@arctrix.com> Signed-off-by: Xin Yang <xyangx@amazon.com> Signed-off-by: yuwenzho <yuwen.zhou@intel.com> Signed-off-by: Yuwen Zhou <yuwen.zhou@intel.com> Signed-off-by: Luciano Martins <lucianomartins@google.com> Signed-off-by: Woosuk Kwon <woosuk@inferact.ai> Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> Signed-off-by: thomasmaindron <thomasmaindron@users.noreply.github.com> Signed-off-by: zjy0516 <riverclouds.zhu@qq.com> Signed-off-by: Netanel Haber <nhaber@nvidia.com> Signed-off-by: Itay Alroy <ialroy@nvidia.com> Signed-off-by: JartX <sagformas@epdcenter.es> Signed-off-by: xiaoming <1259730330@qq.com> Signed-off-by: labAxiaoming <34019940+labAxiaoming@users.noreply.github.com> Signed-off-by: Alex Brooks <albrooks@redhat.com> Signed-off-by: Qiang Li <qiang.li2@amd.com> Signed-off-by: Zhang Jian <jianmusings@gmail.com> Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com> Signed-off-by: ignaciosica <mignacio.sica@gmail.com> Signed-off-by: Angela Yi <angelayi@meta.com> Signed-off-by: Chenguang ZHENG <645327136@qq.com> Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com> Signed-off-by: Dao Le <Dao007forever@gmail.com> Signed-off-by: omerpaz95 <omerpaz95@gmail.com> Signed-off-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com> Signed-off-by: SouthWest7 <am1ao@qq.com> Signed-off-by: Xinan Miao <1403572259@qq.com> Signed-off-by: Yifan Qiao <yifanqiao@inferact.ai> Signed-off-by: qizixi <zixi@inferact.ai> Signed-off-by: Jee Jee Li <pandaleefree@gmail.com> Signed-off-by: Yongye Zhu <zyy1102000@gmail.com> Signed-off-by: Zhewen Li <zhewenli@inferact.ai> Signed-off-by: youkaichao <youkaichao@gmail.com> Signed-off-by: zixi-qi <zixi@inferact.ai> Signed-off-by: wzhao18 <wzhao18.sz@gmail.com> Signed-off-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com> Signed-off-by: Thien Tran <gau.nernst@yahoo.com.sg> Signed-off-by: khluu <khluu000@gmail.com> Signed-off-by: ghphotoframe <854746559@qq.com> Signed-off-by: Isotr0py <Isotr0py@outlook.com> Signed-off-by: Rohan138 <rohanpotdar138@gmail.com> Signed-off-by: zero-uuuuk <bike481227@gmail.com> Co-authored-by: Yuan Tang <terrytangyuan@gmail.com> Co-authored-by: Tao He <linzhu.ht@alibaba-inc.com> Co-authored-by: Chaojun Zhang <chaojun.zhang@intel.com> Co-authored-by: Kunshang Ji <kunshang.ji@intel.com> Co-authored-by: Julien Denize <40604584+juliendenize@users.noreply.github.com> Co-authored-by: velonica0 <47554626+velonica0@users.noreply.github.com> Co-authored-by: San-Nguyen <san.nguyen@ibm.com> Co-authored-by: wuyingjun <71485611@qq.com> Co-authored-by: milesial <milesial@users.noreply.github.com> Co-authored-by: Andreas Karatzas <akaratza@amd.com> Co-authored-by: nemanjaudovic <152565955+nemanjaudovic@users.noreply.github.com> Co-authored-by: Fadi Arafeh <115173828+fadara01@users.noreply.github.com> Co-authored-by: Or Ozeri <oro@il.ibm.com> Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com> Co-authored-by: Ilya Markov <markovilya197@gmail.com> Co-authored-by: zhanqiuhu <49648934+ZhanqiuHu@users.noreply.github.com> Co-authored-by: Yan Ma <yan.ma@intel.com> Co-authored-by: aleksandaryanakiev <alexander.yanakiev@cleverpine.com> Co-authored-by: Aleksandar Yanakiev <alexander.yanakiev@discretestack.com> Co-authored-by: Galigator <Galigator@users.noreply.github.com> Co-authored-by: kse <kevin.sejourne@cloud-temple.com> Co-authored-by: Vasiliy Kuznetsov <vkuzo@users.noreply.github.com> Co-authored-by: Sage Moore <sage@neuralmagic.com> Co-authored-by: larryli2-amd <larryli2@amd.com> Co-authored-by: TJian <tunjian.tan@embeddedllm.com> Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com> Co-authored-by: Rita Brugarolas <Rita.BrugarolasBrufau@amd.com> Co-authored-by: Hashem Hashemi <159079214+amd-hhashemi@users.noreply.github.com> Co-authored-by: bnellnm <49004751+bnellnm@users.noreply.github.com> Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com> Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com> Co-authored-by: Frederik Gossen <frgossen@meta.com> Co-authored-by: bai <17348+bai@users.noreply.github.com> Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com> Co-authored-by: Theresa Shan <Theresa.Shan@amd.com> Co-authored-by: Theresa Shan <thshan@smci355-ccs-aus-n08-21.prov.aus.ccs.cpe.ice.amd.com> Co-authored-by: Cao Qian <talexcao@gmail.com> Co-authored-by: Matthew Bonanni <mbonanni@redhat.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Yanan Cao <gmagogsfm@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Misa <61035833+misaAle@users.noreply.github.com> Co-authored-by: Misael Casarez <misacasa@amazon.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com> Co-authored-by: Luciano Martins <lucianomartins@google.com> Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com> Co-authored-by: Shanshan Shen <467638484@qq.com> Co-authored-by: Simon Mo <simon.mo@hey.com> Co-authored-by: SeongJun Lee <lesj0610@gmail.com> Co-authored-by: hangy-amd <Hang.Yang@amd.com> Co-authored-by: Zeyu Zhang <111627783+Alchuang22-dev@users.noreply.github.com> Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io> Co-authored-by: Jhao-Ting Chen <jhaotingc@nvidia.com> Co-authored-by: Kris Hung <krish@nvidia.com> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: artem-spector <artem_spector@yahoo.com> Co-authored-by: artemspector <artems@il.ibm.com> Co-authored-by: Talor Abramovich <talora@nvidia.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Yusuf Mohammad <79484377+YM2132@users.noreply.github.com> Co-authored-by: ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 <hollowman@opensuse.org> Co-authored-by: xiangdong <40376367+zxd1997066@users.noreply.github.com> Co-authored-by: roikoren755 <26850796+roikoren755@users.noreply.github.com> Co-authored-by: Vadim Gimpelson <156319763+vadiklyutiy@users.noreply.github.com> Co-authored-by: Nick Hill <nickhill123@gmail.com> Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com> Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn> Co-authored-by: Roger Wang <hey@rogerw.io> Co-authored-by: Zijing Liu <liuzijing2014@users.noreply.github.com> Co-authored-by: Rishi Puri <riship@nvidia.com> Co-authored-by: Stefano Castagnetta <scastagnetta@nvidia.com> Co-authored-by: Benjamin Chislett <bchislett@nvidia.com> Co-authored-by: Fergus <fergus.barratt00@gmail.com> Co-authored-by: Jakub Zakrzewski <jzakrzewski@nvidia.com> Co-authored-by: Khushali Desai <kdesai@extremenetworks.com> Co-authored-by: rasmith <Randall.Smith@amd.com> Co-authored-by: Soila Kavulya <soila.p.kavulya@intel.com> Co-authored-by: EdalatiAli <aliedalati@cohere.com> Co-authored-by: Jaseel Muhammad <jaseel.muhammad@mbzuai.ac.ae> Co-authored-by: Isotr0py <2037008807@qq.com> Co-authored-by: Martin Hickey <martin.hickey@ie.ibm.com> Co-authored-by: Rishapveer Singh <singhrishapveer@gmail.com> Co-authored-by: Carl Y <4531192+carlyou@users.noreply.github.com> Co-authored-by: Micah Williamson <micah.williamson@amd.com> Co-authored-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com> Co-authored-by: Bugen Zhao <i@bugenzhao.com> Co-authored-by: philip-essential <169196560+philip-essential@users.noreply.github.com> Co-authored-by: storyicon <storyicon@foxmail.com> Co-authored-by: Johnny Yang <24908445+jcyang43@users.noreply.github.com> Co-authored-by: AllenDou <allen.dou@hotmail.com> Co-authored-by: zixiao <shunli.dsl@alibaba-inc.com> Co-authored-by: lyd1992 <105697319+lyd1992@users.noreply.github.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com> Co-authored-by: xiao <102247755+Wangxiaoxiaoa@users.noreply.github.com> Co-authored-by: Wangxiaoxiaoa <Wangxiaoxiaoa@users.noreply.github.com> Co-authored-by: Hank_ <37239608+ILikeIneine@users.noreply.github.com> Co-authored-by: fxmarty-amd <felmarty@amd.com> Co-authored-by: Kyle Sayers <kylesayrs@gmail.com> Co-authored-by: Angela Yi <yiangela7@gmail.com> Co-authored-by: Fynn Schmitt-Ulms <fschmitt@redhat.com> Co-authored-by: Doug Smith <dosmith@redhat.com> Co-authored-by: Flora Feng <4florafeng@gmail.com> Co-authored-by: Lucas Kabela <lucaskabela@meta.com> Co-authored-by: Honglin Cao <Caohonglin317@hotmail.com> Co-authored-by: Simon Danielsson <70206058+simondanielsson@users.noreply.github.com> Co-authored-by: Matvei Pashkovskii <mpashkov@amd.com> Co-authored-by: Julian Huang <huangzhilin.hzl@gmail.com> Co-authored-by: 墨楼 <huangzhilin.hzl@antgroup.com> Co-authored-by: Codex <codex@openai.com> Co-authored-by: Robert Shaw <robertgshaw2@gmail.com> Co-authored-by: Zhengxu Chen <zhxchen17@fb.com> Co-authored-by: Shengqi Chen <harry-chen@outlook.com> Co-authored-by: liuzhenwei <zhenweiliu@habana.ai> Co-authored-by: Matthias Gehre <matthias.gehre@amd.com> Co-authored-by: BadrBasowid <61441185+BadrBasowid@users.noreply.github.com> Co-authored-by: almayne <anna.mayne@arm.com> Co-authored-by: Fadi Arafeh <fadi.arafeh@arm.com> Co-authored-by: Li, Jiang <jiang1.li@intel.com> Co-authored-by: Richard Zou <zou3519@users.noreply.github.com> Co-authored-by: stevenkuang <stevenkuang@tencent.com> Co-authored-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com> Co-authored-by: pschlan-amd <pschlan@amd.com> Co-authored-by: shaharmor98 <17088876+shaharmor98@users.noreply.github.com> Co-authored-by: Jackmin801 <56836461+Jackmin801@users.noreply.github.com> Co-authored-by: Johnny <johnnynuca14@gmail.com> Co-authored-by: czhu-cohere <conway.zhu@cohere.com> Co-authored-by: Sophie du Couédic <sop@zurich.ibm.com> Co-authored-by: Hemanth Acharya <Hemanth.Acharya@amd.com> Co-authored-by: Doug Campos <qmx@qmx.me> Co-authored-by: Dmitry Tokarev <dtokarev@nvidia.com> Co-authored-by: yzong-rh <yzong@redhat.com> Co-authored-by: timon0305 <timon0305@outlook.com> Co-authored-by: Vinayak Kumar <46377914+VinayakMishra95@users.noreply.github.com> Co-authored-by: Neil Schemenauer <nas-github@arctrix.com> Co-authored-by: Xin Yang <105740670+xyang16@users.noreply.github.com> Co-authored-by: Yuwen Zhou <yuwen.zhou@intel.com> Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu> Co-authored-by: Thomas <153741656+thomasmaindron@users.noreply.github.com> Co-authored-by: thomasmaindron <thomasmaindron@users.noreply.github.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Co-authored-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com> Co-authored-by: Itay Alroy <75032521+itayalroy@users.noreply.github.com> Co-authored-by: Jinzhen Lin <jinzhen.ljz@antgroup.com> Co-authored-by: JartX <sagformas@epdcenter.es> Co-authored-by: labAxiaoming <34019940+labAxiaoming@users.noreply.github.com> Co-authored-by: Alex Brooks <albrooks@redhat.com> Co-authored-by: qli88 <qiang.li2@amd.com> Co-authored-by: Zhang Jian <jianmusings@gmail.com> Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com> Co-authored-by: Ignacio Sica <mignacio.sica@gmail.com> Co-authored-by: Angela Yi <angelayi@meta.com> Co-authored-by: Chenguang Zheng <645327136@qq.com> Co-authored-by: Dao007forever <dao007forever@gmail.com> Co-authored-by: omerpaz95 <73347585+omerpaz95@users.noreply.github.com> Co-authored-by: Or Ozeri <or@ozery.com> Co-authored-by: Xinan Miao <1403572259@qq.com> Co-authored-by: SouthWest7 <am1ao@qq.com> Co-authored-by: Wang Xingran <72983099+wangxingran222@users.noreply.github.com> Co-authored-by: Yifan Qiao <yifanqiao@inferact.ai> Co-authored-by: Yongye Zhu <zyy1102000@gmail.com> Co-authored-by: Yongye Zhu <yongye@inferact.ai> Co-authored-by: Simon Mo <simon@inferact.ai> Co-authored-by: Giancarlo Delfin <gdelfin@inferact.ai> Co-authored-by: Roy Wang <yasong.wang@inferact.ai> Co-authored-by: Woosuk Kwon <woosuk@inferact.ai> Co-authored-by: youkaichao <youkaichao@gmail.com> Co-authored-by: Zhewen Li <jerven.vllm@gmail.com> Co-authored-by: Zijing Liu <liuzijing2014@gmail.com> Co-authored-by: khluu <khluu000@gmail.com> Co-authored-by: qizixi <zixi@inferact.ai> Co-authored-by: Zhewen Li <zhewenli@inferact.ai> Co-authored-by: Zhewen Li <zhewenli@meta.com> Co-authored-by: qizixi <22851944+zixi-qi@users.noreply.github.com> Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com> Co-authored-by: Wei Zhao (Engrg-Hardware 1) <weizha@login-bia02.bia.clusters.nvidia.com> Co-authored-by: Thien Tran <gau.nernst@yahoo.com.sg> Co-authored-by: JiangWeixiang <854746559@qq.com> Co-authored-by: Rohan Potdar <66227218+Rohan138@users.noreply.github.com> Co-authored-by: Copilot <copilot@github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enabling new 1 GPU MI300X test agents
cc @kenroche