[MoE Refactor] Move the shared/fused expert output sum into MoERunnerBase

[bnellnm]

Purpose

• Move the summation of the shared and fused expert outputs into MoERunnerBase and cleanup the final shared/fused all reduce code. Remove corresponding code from models that use SharedFusedMoE
• Move the final TP reduce code into MoERunnerBase.

Test Plan

CI MoE refactor tests

Run gsm8k evals on the following models:

• arcee-ai/Trinity-Mini
• deepseek-ai/DeepSeek-R1
• google/gemma-4-26B-A4B-it
• zai-org/GLM-4.7-Flash
• openai/gpt-oss-20b
• ibm-granite/granite-4.0-h-small
• ai21labs/AI21-Jamba2-Mini
• LiquidAI/LFM2.5-350M
• meta-llama/Llama-4-Scout-17B-16E-Instruct
• MiniMaxAI/MiniMax-M2.7
• mistralai/Mixtral-8x7B-v0.1
• nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4
• allenai/OLMoE-1B-7B-0125-Instruct
• microsoft/Phi-tiny-MoE-instruct
• sarvamai/sarvam-30b
• stepfun-ai/Step-3.5-Flash

Test Result

Model	Baseline	PR
arcee-ai/Trinity-Mini	0.8408	0.8476
deepseek-ai/DeepSeek-R1	0.9492	0.9522
google/gemma-4-26B-A4B-it	0.3017	0.3207
zai-org/GLM-4.7-Flash	0.8241	0.8393
openai/gpt-oss-20b	0.3154	0.3002
ibm-granite/granite-4.0-h-small	0.8400	0.8438
ai21labs/AI21-Jamba2-Mini	0.7665	0.7635
LiquidAI/LFM2.5-350M	0.2092	0.2153
meta-llama/Llama-4-Scout-17B-16E-Instruct	TIMEOUT	TIMEOUT
MiniMaxAI/MiniMax-M2.7	0.9249	0.9242
mistralai/Mixtral-8x7B-v0.1	0.5512	0.5701
nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4	0.9295	0.9333
allenai/OLMoE-1B-7B-0125-Instruct	0.6770	0.6907
microsoft/Phi-tiny-MoE-instruct	0.7020	0.7028
sarvamai/sarvam-30b	0.6588	0.6603
stepfun-ai/Step-3.5-Flash	FAIL	FAIL
baidu/ERNIE-4.5-VL-28B-A3B-Thinking	0.7506	0.7536

lm-eval RedHatAI/DeepSeek-Coder-V2-Lite-Instruct-FP8 with SP

main:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.7551|±  |0.0118|
|     |       |strict-match    |     5|exact_match|↑  |0.7407|±  |0.0121|

PR:

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.7582|±  |0.0118|
|     |       |strict-match    |     5|exact_match|↑  |0.7392|±  |0.0121|

cc @robertgshaw2-redhat , @tlrmchlsmth , @yzong-rh

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @bnellnm.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[gemini-code-assist]

Code Review

This pull request introduces a significant and well-designed refactoring of the Mixture-of-Experts (MoE) implementation. The introduction of the MoERunner abstraction effectively separates the MoE execution logic from the layer definition, improving modularity and maintainability. The changes across various model files to adapt to this new abstraction are consistent and simplify the model-specific code. The addition of support for zero experts and the corresponding tests are also valuable.

However, I've identified a critical issue in vllm/model_executor/models/transformers/moe.py where a removed parameter is still being passed, which will lead to a runtime error. Additionally, there's a potential regression in the same file regarding the detection of shared experts, which now relies solely on parameter names instead of also checking the model configuration. Please see the detailed comments for suggestions on how to address these issues.

_{Note: Security Review did not run due to the size of the PR.}