added parser for moe detection with test by morrison-turnansky · Pull Request #1 · morrison-turnansky/vllm

morrison-turnansky · 2025-10-14T20:29:43Z

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

ProExpertProg · 2025-10-15T00:16:22Z

vllm/config/model.py

+            "num_experts",  # Jamba
+            "moe_num_experts",  # Dbrx
+            "n_routed_experts",  # DeepSeek
+            "num_local_experts",  # Mixtral


Qwen and LLama4?

I have set things up so that we can add proper keys for the various schemas. We can keep a list and add before we merge.

track on draft MR since we don't have to worry about the clean diff anymore. vllm-project#26847 (comment)

vllm/config/vllm.py

…le access (vllm-project#26810) Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>

Signed-off-by: Zhikaiiii <1658973216@qq.com>

…e' in get_kv_connector_stats (vllm-project#26851) Signed-off-by: Chendi Xue <chendi.xue@intel.com>

Signed-off-by: mgoin <mgoin64@gmail.com>

): name change compilation level to compilation mode, deprecation compilation level (vllm-project#26355) Signed-off-by: morrison-turnansky <mturnans@redhat.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

Signed-off-by: Kourosh Hakhamaneshi <Kourosh@anyscale.com>

…t#24972) (vllm-project#25589) Signed-off-by: taohui <taohui3@gmail.com> Signed-off-by: Tao Hui <taohui3@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

Signed-off-by: windsonsea <haifeng.yao@daocloud.io>

…llm-project#26773) Signed-off-by: izhuhaoran <izhuhaoran@qq.com> Co-authored-by: Michael Goin <mgoin64@gmail.com>

…h serve (vllm-project#26784) Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>

Signed-off-by: angelayi <yiangela7@gmail.com>

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>

Signed-off-by: Boyuan Feng <boyuan@meta.com>

…ect#26873) Signed-off-by: sanghol <sanghol@allenai.org>

…t backends (vllm-project#26656) Signed-off-by: MengqingCao <cmq0113@163.com>

…sts (vllm-project#26875) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: angelayi <yiangela7@gmail.com>

…roject#26456) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>

…llm-project#21810) Signed-off-by: Ye (Charlotte) Qi <yeq@meta.com> Signed-off-by: zhewenli <zhewenli@meta.com> Co-authored-by: Ye (Charlotte) Qi <yeq@meta.com> Co-authored-by: Ye (Charlotte) Qi <ye.charlotte.qi@gmail.com>

Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: youkaichao <youkaichao@gmail.com>

…sed by Flash Linear Attention ops. (vllm-project#26853) Co-authored-by: Xudong Ma <mxd@meta.com>

…e in bailing_moe (vllm-project#26876) Signed-off-by: vito.yy <vito.yy@antgroup.com>

…project#26842) Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

…ect#26319) Signed-off-by: li2haipeng <44383182+li2haipeng@users.noreply.github.com> Signed-off-by: Haipeng Li <li2haipeng@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

vllm-project#25370) Signed-off-by: wang.yuqi <noooop@126.com>

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

Signed-off-by: adabeyta <aabeyta@redhat.com>

…for compilationmode.none

… fields controlled by optimization level are forced to be None Signed-off-by: morrison-turnansky <mturnans@redhat.com>

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

ProExpertProg reviewed Oct 15, 2025

View reviewed changes

morrison-turnansky force-pushed the issue-20283-level branch from 211fe4a to 4a53a64 Compare October 15, 2025 00:28

morrison-turnansky force-pushed the issue-20283-model-config branch from 9c6d2e0 to 930fd18 Compare October 15, 2025 00:53

ProExpertProg reviewed Oct 15, 2025

View reviewed changes

vllm/config/vllm.py Outdated Show resolved Hide resolved

Jialin and others added 26 commits October 15, 2025 01:41

[Core][Easy] Use envs.__getattr__ for all Unify to environment variab…

07ca70a

…le access (vllm-project#26810) Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>

[Bugfix]fix Qwen3 xml tool parser (vllm-project#26345)

9354660

Signed-off-by: Zhikaiiii <1658973216@qq.com>

[BUGFIX][NIXL] quick fix for 'assert self.connector_worker is not Non…

bfad142

…e' in get_kv_connector_stats (vllm-project#26851) Signed-off-by: Chendi Xue <chendi.xue@intel.com>

Disable FlashInfer sampler by default (vllm-project#26859)

e66d787

Signed-off-by: mgoin <mgoin64@gmail.com>

[Bugfix] Fixes prefix-repetition benchmark script (vllm-project#26828)

a2986b3

Signed-off-by: Kourosh Hakhamaneshi <Kourosh@anyscale.com>

[Docs] Move build.inc into arm.inc (vllm-project#26862)

c43ca82

Signed-off-by: windsonsea <haifeng.yao@daocloud.io>

[CI/Build][Bugfix] fix qutlass cmake error when set QUTLASS_SRC_DIR (v…

e471d7c

…llm-project#26773) Signed-off-by: izhuhaoran <izhuhaoran@qq.com> Co-authored-by: Michael Goin <mgoin64@gmail.com>

[Feature] default --extra-body param to disable thinking in vllm benc…

a27b288

…h serve (vllm-project#26784) Signed-off-by: rongfu.leng <rongfu.leng@daocloud.io>

[BugFix] Patch inductor partitioning logic (vllm-project#26735)

7cfa420

Signed-off-by: angelayi <yiangela7@gmail.com>

[Bugfix] Fix qwen3-omni audio truncation issue (vllm-project#26815)

8c851f6

Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>

[Graph Partition] pass tests for decorator (vllm-project#26831)

f0862ea

Signed-off-by: Boyuan Feng <boyuan@meta.com>

[Bugfix][Multi Modal] Fix incorrect Molmo token processing (vllm-proj…

8865da1

…ect#26873) Signed-off-by: sanghol <sanghol@allenai.org>

[DSA][MLA] Tiny refactor on DeepSeek to make it reusable for differen…

302ef40

…t backends (vllm-project#26656) Signed-off-by: MengqingCao <cmq0113@163.com>

[Misc] Use helper function to generate dummy messages in OpenAI MM te…

b8a4572

…sts (vllm-project#26875) Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[bugfix] Lazy import cv2 (vllm-project#26869)

efdef57

Signed-off-by: angelayi <yiangela7@gmail.com>

[Deepseek-V3.2][Kernel] Integrate cuda indexer k cache gather (vllm-p…

f5ed68e

…roject#26456) Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>

[CI] Fix mypy for vllm/executor (vllm-project#26845)

71557a5

Signed-off-by: yewentao256 <zhyanwentao@126.com>

[Doc] ruff format remaining Python examples (vllm-project#26795)

6256697

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

[doc] add Context Parallel Deployment doc (vllm-project#26877)

650b51f

Signed-off-by: youkaichao <youkaichao@gmail.com>

[Misc] Update TritonLanguagePlaceholder to have attributes that are u…

5210dc3

…sed by Flash Linear Attention ops. (vllm-project#26853) Co-authored-by: Xudong Ma <mxd@meta.com>

[Fix] Remove divisibility requirement between num_kv_heads and tp_siz…

5c3bae1

…e in bailing_moe (vllm-project#26876) Signed-off-by: vito.yy <vito.yy@antgroup.com>

[Easy] Get rid of unnecessary paraenthesis in kv_cache_manager (vllm-…

7f83b4e

…project#26842) Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>

[Platform] allow platform to init dp group (vllm-project#22243)

db1764e

Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>

li2haipeng and others added 2 commits October 15, 2025 09:45

[Lora]Load tuned multi-lora kernel configs from json files (vllm-proj…

d4d1a60

…ect#26319) Signed-off-by: li2haipeng <44383182+li2haipeng@users.noreply.github.com> Signed-off-by: Haipeng Li <li2haipeng@gmail.com> Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>

[Model][2/N] Improve all pooling task | Support multi-vector retrieval (

f54f851

vllm-project#25370) Signed-off-by: wang.yuqi <noooop@126.com>

morrison-turnansky force-pushed the issue-20283-model-config branch from 930fd18 to e849607 Compare October 15, 2025 12:04

morrison-turnansky and others added 3 commits October 15, 2025 13:06

added parser for moe detection with test

4cf0141

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

Set up -O infrastrucutre

c264394

Signed-off-by: adabeyta <aabeyta@redhat.com>

name change and removed editing backedn in _apply_optimization_level …

4c8f770

…for compilationmode.none

morrison-turnansky force-pushed the issue-20283-model-config branch from e849607 to 4c8f770 Compare October 15, 2025 13:07

updated defaults for each pass config

3b1c862

morrison-turnansky force-pushed the issue-20283-model-config branch from 4093a43 to 3b1c862 Compare October 15, 2025 13:54

morrison-turnansky added 5 commits October 15, 2025 14:21

set cuda graph mode defaults

49e52fd

added skelaton for non model specifc settings, and test to veriy that…

2ee0cb8

… fields controlled by optimization level are forced to be None Signed-off-by: morrison-turnansky <mturnans@redhat.com>

made is_model_moe inaccessible from user

f33d830

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

added parsing function to determine if model is quantized

3b6d03f

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

added model specific optimizations

23985af

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

morrison-turnansky force-pushed the issue-20283-model-config branch from 69e2ccb to 23985af Compare October 15, 2025 17:53

morrison-turnansky added 2 commits October 15, 2025 20:21

updated default config design

0719123

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

added vllm config default test

74d5a6a

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

morrison-turnansky closed this Oct 16, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

added parser for moe detection with test#1

added parser for moe detection with test#1
morrison-turnansky wants to merge 39 commits intoissue-20283-levelfrom
issue-20283-model-config

morrison-turnansky commented Oct 14, 2025

Uh oh!

ProExpertProg Oct 15, 2025

Uh oh!

morrison-turnansky Oct 15, 2025

Uh oh!

morrison-turnansky Oct 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

morrison-turnansky commented Oct 14, 2025

Purpose

Test Plan

Test Result

Uh oh!

ProExpertProg Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

morrison-turnansky Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

morrison-turnansky Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants