[MoE Refactor] Rename FusedMoE to FusedMoEFactory - #44941
Merged
Conversation
Signed-off-by: Bill Nell <bnell@redhat.com>
mergify
Bot
added
deepseek
bnellnm
marked this pull request as ready for review
June 10, 2026 18:01
bnellnm
requested review from
22quinn,
AndreasKaratzas,
ProExpertProg,
WoosukKwon,
hmellor,
houseroad,
jeejeelee,
mgoin,
patrickvonplaten,
pavanimajety,
robertgshaw2-redhat,
sighingnow,
tlrmchlsmth,
tomeras91,
vadiklyutiy,
yewentao256,
youkaichao and
zyongye
as code owners
June 10, 2026 18:01
Signed-off-by: Bill Nell <bnell@redhat.com>
github-project-automation
Bot
moved this from Ready
to Done
in gpt-oss Issues & Enhancements
pawel-olejniczak
added a commit
to pawel-olejniczak/vllm-gaudi
that referenced
this pull request
Aug 3, 2026
…MoE imports to upstream FusedMoE factory rename and Marlin removal Root cause: vLLM PR #44941 (1) renamed the MoE factory FusedMoE -> FusedMoEFactory and (2) split compressed_tensors_moe into a package, removing the compressed_tensors_moe_wna16_marlin submodule and folding CompressedTensorsWNA16MarlinMoEMethod into the unified CompressedTensorsWNA16MoEMethod. The old hard imports broke vllm_gaudi ops import at load, cascading to all 59 hourly jobs. Upstream: vllm-project/vllm#44941 Fix: import the current upstream symbols directly -- FusedMoEFactory (aliased locally to FusedMoE) and the unified CompressedTensorsWNA16MoEMethod -- and drop the standalone wna16_marlin module import and its monkeypatch. We pin to a single vLLM revision, so no pre-#44941 fallback is kept. Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
pawel-olejniczak
added a commit
to pawel-olejniczak/vllm-gaudi
that referenced
this pull request
Aug 3, 2026
…FusedMoEFactory to FusedMoE after upstream rename Root cause: upstream vLLM PR #44941 renamed the MoE factory function FusedMoE -> FusedMoEFactory in fused_moe.layer; the old name is gone, so vllm-gaudi's load-time imports raise ImportError, cascading to all 59 CI jobs. Upstream: vllm-project/vllm#44941 Fix: import FusedMoEFactory as FusedMoE via a try/except shim at the 5 ops/model import sites (version-agnostic across the rename boundary). Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
pawel-olejniczak
added a commit
to pawel-olejniczak/vllm-gaudi
that referenced
this pull request
Aug 3, 2026
…MoE imports to upstream FusedMoE factory rename and Marlin removal Root cause: vLLM PR #44941 (1) renamed the MoE factory FusedMoE -> FusedMoEFactory and (2) split compressed_tensors_moe into a package, removing the compressed_tensors_moe_wna16_marlin submodule and folding CompressedTensorsWNA16MarlinMoEMethod into the unified CompressedTensorsWNA16MoEMethod. The old hard imports broke vllm_gaudi ops import at load, cascading to all 59 hourly jobs. Upstream: vllm-project/vllm#44941 Fix: import the current upstream symbols directly -- FusedMoEFactory (aliased locally to FusedMoE) and the unified CompressedTensorsWNA16MoEMethod -- and drop the standalone wna16_marlin module import and its monkeypatch. We pin to a single vLLM revision, so no pre-#44941 fallback is kept. Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
pawel-olejniczak
added a commit
to pawel-olejniczak/vllm-gaudi
that referenced
this pull request
Aug 3, 2026
…FusedMoEFactory to FusedMoE after upstream rename Root cause: upstream vLLM PR #44941 renamed the MoE factory function FusedMoE -> FusedMoEFactory in fused_moe.layer; the old name is gone, so vllm-gaudi's load-time imports raise ImportError, cascading to all 59 CI jobs. Upstream: vllm-project/vllm#44941 Fix: import FusedMoEFactory as FusedMoE via a try/except shim at the 5 ops/model import sites (version-agnostic across the rename boundary). Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
pawel-olejniczak
added a commit
to pawel-olejniczak/vllm-gaudi
that referenced
this pull request
Aug 3, 2026
…MoE imports to upstream FusedMoE factory rename and Marlin removal Root cause: vLLM PR #44941 (1) renamed the MoE factory FusedMoE -> FusedMoEFactory and (2) split compressed_tensors_moe into a package, removing the compressed_tensors_moe_wna16_marlin submodule and folding CompressedTensorsWNA16MarlinMoEMethod into the unified CompressedTensorsWNA16MoEMethod. The old hard imports broke vllm_gaudi ops import at load, cascading to all 59 hourly jobs. Upstream: vllm-project/vllm#44941 Fix: import the current upstream symbols directly -- FusedMoEFactory (aliased locally to FusedMoE) and the unified CompressedTensorsWNA16MoEMethod -- and drop the standalone wna16_marlin module import and its monkeypatch. We pin to a single vLLM revision, so no pre-#44941 fallback is kept. Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
pawel-olejniczak
added a commit
to pawel-olejniczak/vllm-gaudi
that referenced
this pull request
Aug 4, 2026
…FusedMoEFactory to FusedMoE after upstream rename Root cause: upstream vLLM PR #44941 renamed the MoE factory function FusedMoE -> FusedMoEFactory in fused_moe.layer; the old name is gone, so vllm-gaudi's load-time imports raise ImportError, cascading to all 59 CI jobs. Upstream: vllm-project/vllm#44941 Fix: import FusedMoEFactory as FusedMoE via a try/except shim at the 5 ops/model import sites (version-agnostic across the rename boundary). Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
pawel-olejniczak
added a commit
to pawel-olejniczak/vllm-gaudi
that referenced
this pull request
Aug 4, 2026
…MoE imports to upstream FusedMoE factory rename and Marlin removal Root cause: vLLM PR #44941 (1) renamed the MoE factory FusedMoE -> FusedMoEFactory and (2) split compressed_tensors_moe into a package, removing the compressed_tensors_moe_wna16_marlin submodule and folding CompressedTensorsWNA16MarlinMoEMethod into the unified CompressedTensorsWNA16MoEMethod. The old hard imports broke vllm_gaudi ops import at load, cascading to all 59 hourly jobs. Upstream: vllm-project/vllm#44941 Fix: import the current upstream symbols directly -- FusedMoEFactory (aliased locally to FusedMoE) and the unified CompressedTensorsWNA16MoEMethod -- and drop the standalone wna16_marlin module import and its monkeypatch. We pin to a single vLLM revision, so no pre-#44941 fallback is kept. Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
iboiko-habana
pushed a commit
to vllm-project/vllm-gaudi
that referenced
this pull request
Aug 5, 2026
…sedMoE import after upstream factory rename and Marlin removal (#1691) ## Root cause Upstream vLLM renamed the fused-MoE factory function `FusedMoE` to `FusedMoEFactory` in `vllm.model_executor.layers.fused_moe.layer` and removed the `compressed_tensors_moe_wna16_marlin` submodule/class. The Gaudi plugin's ops module imports the old `FusedMoE` symbol at load time, so the import fails and the failure cascades to every CI job (all 59 red). ## Upstream PR vllm-project/vllm#44941 Renamed the MoE factory entry point and dropped the WNA16 Marlin MoE method. ## Fix Re-export a `FusedMoEFactory` alias for the renamed symbol so the ops import resolves against both old and new upstream layouts, and guard the WNA16 Marlin MoE import so its upstream removal no longer aborts the ops module load. --------- Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
wangxiyuan
pushed a commit
to vllm-project/vllm-ascend
that referenced
this pull request
Aug 11, 2026
### What this PR does / why we need it? #### Upgrade baseline - Update the verified vLLM main anchor from [`2e09247c2d7b6b97d13af6e71a85bf8d1271deb6`](vllm-project/vllm@2e09247) to [`58d3918e3ea0a544ffedadad2ba84559e9c51d8f`](vllm-project/vllm@58d3918). The full upstream range is available in this [comparison](vllm-project/vllm@2e09247...58d3918). - Preserve the vLLM `0.26.0` compatibility lane while adapting the main lane to the new upstream contracts. Version gates use `vllm_version_is("0.26.0")` and are limited to real contract differences. - The changes are organized in the same order as the changed files in this PR. Each item identifies the upstream change, the downstream adaptation, and why the adaptation is required. #### Changes by file ##### 1. `.github/vllm-main-verified.commit` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Update anchor to `58d3918e` | Upgrade window [0351e9aa...58d3918e](vllm-project/vllm@0351e9a...58d3918). | Set anchor. | Source of truth for main2main workflow. | ##### 2. `vllm_ascend/patch/platform/patch_fused_moe.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate FusedMoE → FusedMoEFactory rename | [vllm #44941](vllm-project/vllm#44941) renamed `FusedMoE` to `FusedMoEFactory`. | On main, capture and patch `FusedMoEFactory`; on v0.26.0, also patch legacy `FusedMoE`. | Both lanes need the Ascend runner patch at the correct binding. | ##### 3. `vllm_ascend/models/deepseek_v4.py` / `vllm_ascend/models/minimax_m3/minimax_m3.py` / `vllm_ascend/ops/fused_moe/fused_moe.py` / `vllm_ascend/ops/fused_moe/routed_experts.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Import and use `FusedMoEFactory`; remove dead `FusedMoE` re-export | [vllm #44941](vllm-project/vllm#44941). | Replace `FusedMoE` with `FusedMoEFactory`. | Old symbol no longer exists on main. Remove stale `FusedMoE` re-export from `fused_moe.py` and dead reference in `routed_experts.py` comment. | ##### 4. `tests/ut/models/test_deepseek_v4_moe.py` / `tests/ut/models/minimax_m3/test_minimax_m3.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Update monkeypatch target to `FusedMoEFactory` | [vllm #44941](vllm-project/vllm#44941). | `"FusedMoE"` → `"FusedMoEFactory"`. | Must match the symbol imported by models. | ##### 5. `vllm_ascend/worker/model_runner_v1.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate `calculate_kv_scales` removal | [vllm #49389](vllm-project/vllm#49389) removed runtime KV-scale calculation. | Add `vllm_version_is("0.26.0")` guard. | Ascend MRV1 still supports it on v0.26.0; attribute absent on main. | | Version-gate `clear_buffer()` removal | [vllm #50721](vllm-project/vllm#50721) removed `clear_buffer()` from `RoutedExpertsCapturer`. | Wrap in `vllm_version_is("0.26.0")` guard. | On main, each routed layer overwrites current step's token rows. | ##### 6. `vllm_ascend/models/layer/attention/layer.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Remove dead `Q/K/V_SCALE_CONSTANT` references | [vllm #49389](vllm-project/vllm#49389) removed env var registrations. Module-level constants still exist. | Remove unused `q_range`/`k_range`/`v_range` initializations and dead `import envs`. | Dead-code cleanup; `DSAAttention.forward()` never used these attributes. | ##### 7. `tests/ut/patch/platform/test_deepseek_v4_thinking.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate reasoning effort expectations | [vllm #50580](vllm-project/vllm#50580) maps `low`/`minimal`/`medium` → `low`. | `vllm_version_is("0.26.0")` guard. | v0.26.0 keeps old mapping; main uses new. | ##### 8. `tests/e2e/pull_request/one_card/spec_decode/test_extract_hidden_states.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Enable `chunked_prefill` for hybrid model | [vllm #50991](vllm-project/vllm#50991) enabled prefix cache by default for Mamba/hybrid models. | `False` → `True`. | Hybrid model now requires chunked prefill. | ##### 9. `vllm_ascend/patch/platform/patch_vision.py` (new) + `vllm_ascend/patch/platform/__init__.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Patch `FusedInputNorm.forward` eps=0.0 → eps=1e-5 | [vllm #50411](vllm-project/vllm#50411) added `FusedInputNorm` with `F.batch_norm(eps=0.0)`. | Monkey-patch forward to use `eps=1e-5`; guarded with `contextlib.suppress(ImportError)`. | Upstream PyTorch 2.13.0 allows eps >= 0 for inference; vllm-ascend PyTorch 2.10.0 requires eps > 0 always. Release wheels lack `FusedInputNorm`. Remove this patch once bundled PyTorch >= 2.13.0. | ##### 10. `vllm_ascend/ops/triton/mamba/postprocess.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Add kernel signature parameters | [vllm #50432](vllm-project/vllm#50432) changed signature. | Add `CONV_STATE_DIM_FIRST`, `HAS_IDX_MAPPING`, `PRECOMPUTED_NEW_COMPUTED`, `state_dim_row_count/stride`, `idx_mapping_ptr` parameters, and `num_loops` for DS conv copy. | Must match upstream kernel contract. | ##### 11. `tests/e2e/conftest.py` / `tests/ut/spec_decode/test_speculators_vwn_eagle3.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | HunyuanVL placeholder version gate; remove unused `maybe_calc_kv_scales` mock | [vllm #49691](vllm-project/vllm#49691), #49389. | Version gate and dead-mock removal. | Adapt to upstream contract changes. | ##### 12. `vllm_ascend/patch/__init__.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Document FusedMoE → FusedMoEFactory rename and new patch_vision entry | — | Update patch registry documentation. | Keep the patch manifest in sync with reality. | #### Compatibility and review notes - Version gates use `vllm_version_is("0.26.0")` exclusively; no `hasattr` fallbacks beyond the explicitly justified `clear_buffer` guard (where the upstream change is a method removal, not a rename). - The `FusedMoE` → `FusedMoEFactory` rename is applied consistently across all call sites: `deepseek_v4.py`, `minimax_m3.py`, `fused_moe.py`, `routed_experts.py`, and `patch_fused_moe.py`. - The `layer.py` `Q/K/V_SCALE_CONSTANT` removal is a dead-code cleanup: the `DSAAttention` class initialized these tensors from `envs` module-level constants (which still exist), but never used them in `forward()`. ### Does this PR introduce _any_ user-facing change? No. This is a compatibility update; no new Ascend-specific public API is introduced. ### How was this patch tested? CI on the branch. See Buildkite workflow run for detailed results. - vLLM version: v0.26.0 - vLLM main: vllm-project/vllm@0351e9a --------- Signed-off-by: liaoqidan <1107297340@qq.com>
HMCCMH
pushed a commit
to hotTea123/vllm-ascend
that referenced
this pull request
Aug 12, 2026
### What this PR does / why we need it? #### Upgrade baseline - Update the verified vLLM main anchor from [`2e09247c2d7b6b97d13af6e71a85bf8d1271deb6`](vllm-project/vllm@2e09247) to [`58d3918e3ea0a544ffedadad2ba84559e9c51d8f`](vllm-project/vllm@58d3918). The full upstream range is available in this [comparison](vllm-project/vllm@2e09247...58d3918). - Preserve the vLLM `0.26.0` compatibility lane while adapting the main lane to the new upstream contracts. Version gates use `vllm_version_is("0.26.0")` and are limited to real contract differences. - The changes are organized in the same order as the changed files in this PR. Each item identifies the upstream change, the downstream adaptation, and why the adaptation is required. #### Changes by file ##### 1. `.github/vllm-main-verified.commit` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Update anchor to `58d3918e` | Upgrade window [0351e9aa...58d3918e](vllm-project/vllm@0351e9a...58d3918). | Set anchor. | Source of truth for main2main workflow. | ##### 2. `vllm_ascend/patch/platform/patch_fused_moe.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate FusedMoE → FusedMoEFactory rename | [vllm #44941](vllm-project/vllm#44941) renamed `FusedMoE` to `FusedMoEFactory`. | On main, capture and patch `FusedMoEFactory`; on v0.26.0, also patch legacy `FusedMoE`. | Both lanes need the Ascend runner patch at the correct binding. | ##### 3. `vllm_ascend/models/deepseek_v4.py` / `vllm_ascend/models/minimax_m3/minimax_m3.py` / `vllm_ascend/ops/fused_moe/fused_moe.py` / `vllm_ascend/ops/fused_moe/routed_experts.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Import and use `FusedMoEFactory`; remove dead `FusedMoE` re-export | [vllm #44941](vllm-project/vllm#44941). | Replace `FusedMoE` with `FusedMoEFactory`. | Old symbol no longer exists on main. Remove stale `FusedMoE` re-export from `fused_moe.py` and dead reference in `routed_experts.py` comment. | ##### 4. `tests/ut/models/test_deepseek_v4_moe.py` / `tests/ut/models/minimax_m3/test_minimax_m3.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Update monkeypatch target to `FusedMoEFactory` | [vllm #44941](vllm-project/vllm#44941). | `"FusedMoE"` → `"FusedMoEFactory"`. | Must match the symbol imported by models. | ##### 5. `vllm_ascend/worker/model_runner_v1.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate `calculate_kv_scales` removal | [vllm #49389](vllm-project/vllm#49389) removed runtime KV-scale calculation. | Add `vllm_version_is("0.26.0")` guard. | Ascend MRV1 still supports it on v0.26.0; attribute absent on main. | | Version-gate `clear_buffer()` removal | [vllm #50721](vllm-project/vllm#50721) removed `clear_buffer()` from `RoutedExpertsCapturer`. | Wrap in `vllm_version_is("0.26.0")` guard. | On main, each routed layer overwrites current step's token rows. | ##### 6. `vllm_ascend/models/layer/attention/layer.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Remove dead `Q/K/V_SCALE_CONSTANT` references | [vllm #49389](vllm-project/vllm#49389) removed env var registrations. Module-level constants still exist. | Remove unused `q_range`/`k_range`/`v_range` initializations and dead `import envs`. | Dead-code cleanup; `DSAAttention.forward()` never used these attributes. | ##### 7. `tests/ut/patch/platform/test_deepseek_v4_thinking.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate reasoning effort expectations | [vllm #50580](vllm-project/vllm#50580) maps `low`/`minimal`/`medium` → `low`. | `vllm_version_is("0.26.0")` guard. | v0.26.0 keeps old mapping; main uses new. | ##### 8. `tests/e2e/pull_request/one_card/spec_decode/test_extract_hidden_states.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Enable `chunked_prefill` for hybrid model | [vllm #50991](vllm-project/vllm#50991) enabled prefix cache by default for Mamba/hybrid models. | `False` → `True`. | Hybrid model now requires chunked prefill. | ##### 9. `vllm_ascend/patch/platform/patch_vision.py` (new) + `vllm_ascend/patch/platform/__init__.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Patch `FusedInputNorm.forward` eps=0.0 → eps=1e-5 | [vllm #50411](vllm-project/vllm#50411) added `FusedInputNorm` with `F.batch_norm(eps=0.0)`. | Monkey-patch forward to use `eps=1e-5`; guarded with `contextlib.suppress(ImportError)`. | Upstream PyTorch 2.13.0 allows eps >= 0 for inference; vllm-ascend PyTorch 2.10.0 requires eps > 0 always. Release wheels lack `FusedInputNorm`. Remove this patch once bundled PyTorch >= 2.13.0. | ##### 10. `vllm_ascend/ops/triton/mamba/postprocess.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Add kernel signature parameters | [vllm #50432](vllm-project/vllm#50432) changed signature. | Add `CONV_STATE_DIM_FIRST`, `HAS_IDX_MAPPING`, `PRECOMPUTED_NEW_COMPUTED`, `state_dim_row_count/stride`, `idx_mapping_ptr` parameters, and `num_loops` for DS conv copy. | Must match upstream kernel contract. | ##### 11. `tests/e2e/conftest.py` / `tests/ut/spec_decode/test_speculators_vwn_eagle3.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | HunyuanVL placeholder version gate; remove unused `maybe_calc_kv_scales` mock | [vllm #49691](vllm-project/vllm#49691), #49389. | Version gate and dead-mock removal. | Adapt to upstream contract changes. | ##### 12. `vllm_ascend/patch/__init__.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Document FusedMoE → FusedMoEFactory rename and new patch_vision entry | — | Update patch registry documentation. | Keep the patch manifest in sync with reality. | #### Compatibility and review notes - Version gates use `vllm_version_is("0.26.0")` exclusively; no `hasattr` fallbacks beyond the explicitly justified `clear_buffer` guard (where the upstream change is a method removal, not a rename). - The `FusedMoE` → `FusedMoEFactory` rename is applied consistently across all call sites: `deepseek_v4.py`, `minimax_m3.py`, `fused_moe.py`, `routed_experts.py`, and `patch_fused_moe.py`. - The `layer.py` `Q/K/V_SCALE_CONSTANT` removal is a dead-code cleanup: the `DSAAttention` class initialized these tensors from `envs` module-level constants (which still exist), but never used them in `forward()`. ### Does this PR introduce _any_ user-facing change? No. This is a compatibility update; no new Ascend-specific public API is introduced. ### How was this patch tested? CI on the branch. See Buildkite workflow run for detailed results. - vLLM version: v0.26.0 - vLLM main: vllm-project/vllm@0351e9a --------- Signed-off-by: liaoqidan <1107297340@qq.com>
6 tasks
MmMmaru
pushed a commit
to jiaqi-lee/vllm-ascend
that referenced
this pull request
Aug 19, 2026
### What this PR does / why we need it? #### Upgrade baseline - Update the verified vLLM main anchor from [`2e09247c2d7b6b97d13af6e71a85bf8d1271deb6`](vllm-project/vllm@2e09247) to [`58d3918e3ea0a544ffedadad2ba84559e9c51d8f`](vllm-project/vllm@58d3918). The full upstream range is available in this [comparison](vllm-project/vllm@2e09247...58d3918). - Preserve the vLLM `0.26.0` compatibility lane while adapting the main lane to the new upstream contracts. Version gates use `vllm_version_is("0.26.0")` and are limited to real contract differences. - The changes are organized in the same order as the changed files in this PR. Each item identifies the upstream change, the downstream adaptation, and why the adaptation is required. #### Changes by file ##### 1. `.github/vllm-main-verified.commit` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Update anchor to `58d3918e` | Upgrade window [0351e9aa...58d3918e](vllm-project/vllm@0351e9a...58d3918). | Set anchor. | Source of truth for main2main workflow. | ##### 2. `vllm_ascend/patch/platform/patch_fused_moe.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate FusedMoE → FusedMoEFactory rename | [vllm #44941](vllm-project/vllm#44941) renamed `FusedMoE` to `FusedMoEFactory`. | On main, capture and patch `FusedMoEFactory`; on v0.26.0, also patch legacy `FusedMoE`. | Both lanes need the Ascend runner patch at the correct binding. | ##### 3. `vllm_ascend/models/deepseek_v4.py` / `vllm_ascend/models/minimax_m3/minimax_m3.py` / `vllm_ascend/ops/fused_moe/fused_moe.py` / `vllm_ascend/ops/fused_moe/routed_experts.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Import and use `FusedMoEFactory`; remove dead `FusedMoE` re-export | [vllm #44941](vllm-project/vllm#44941). | Replace `FusedMoE` with `FusedMoEFactory`. | Old symbol no longer exists on main. Remove stale `FusedMoE` re-export from `fused_moe.py` and dead reference in `routed_experts.py` comment. | ##### 4. `tests/ut/models/test_deepseek_v4_moe.py` / `tests/ut/models/minimax_m3/test_minimax_m3.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Update monkeypatch target to `FusedMoEFactory` | [vllm #44941](vllm-project/vllm#44941). | `"FusedMoE"` → `"FusedMoEFactory"`. | Must match the symbol imported by models. | ##### 5. `vllm_ascend/worker/model_runner_v1.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate `calculate_kv_scales` removal | [vllm #49389](vllm-project/vllm#49389) removed runtime KV-scale calculation. | Add `vllm_version_is("0.26.0")` guard. | Ascend MRV1 still supports it on v0.26.0; attribute absent on main. | | Version-gate `clear_buffer()` removal | [vllm #50721](vllm-project/vllm#50721) removed `clear_buffer()` from `RoutedExpertsCapturer`. | Wrap in `vllm_version_is("0.26.0")` guard. | On main, each routed layer overwrites current step's token rows. | ##### 6. `vllm_ascend/models/layer/attention/layer.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Remove dead `Q/K/V_SCALE_CONSTANT` references | [vllm #49389](vllm-project/vllm#49389) removed env var registrations. Module-level constants still exist. | Remove unused `q_range`/`k_range`/`v_range` initializations and dead `import envs`. | Dead-code cleanup; `DSAAttention.forward()` never used these attributes. | ##### 7. `tests/ut/patch/platform/test_deepseek_v4_thinking.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate reasoning effort expectations | [vllm #50580](vllm-project/vllm#50580) maps `low`/`minimal`/`medium` → `low`. | `vllm_version_is("0.26.0")` guard. | v0.26.0 keeps old mapping; main uses new. | ##### 8. `tests/e2e/pull_request/one_card/spec_decode/test_extract_hidden_states.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Enable `chunked_prefill` for hybrid model | [vllm #50991](vllm-project/vllm#50991) enabled prefix cache by default for Mamba/hybrid models. | `False` → `True`. | Hybrid model now requires chunked prefill. | ##### 9. `vllm_ascend/patch/platform/patch_vision.py` (new) + `vllm_ascend/patch/platform/__init__.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Patch `FusedInputNorm.forward` eps=0.0 → eps=1e-5 | [vllm #50411](vllm-project/vllm#50411) added `FusedInputNorm` with `F.batch_norm(eps=0.0)`. | Monkey-patch forward to use `eps=1e-5`; guarded with `contextlib.suppress(ImportError)`. | Upstream PyTorch 2.13.0 allows eps >= 0 for inference; vllm-ascend PyTorch 2.10.0 requires eps > 0 always. Release wheels lack `FusedInputNorm`. Remove this patch once bundled PyTorch >= 2.13.0. | ##### 10. `vllm_ascend/ops/triton/mamba/postprocess.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Add kernel signature parameters | [vllm #50432](vllm-project/vllm#50432) changed signature. | Add `CONV_STATE_DIM_FIRST`, `HAS_IDX_MAPPING`, `PRECOMPUTED_NEW_COMPUTED`, `state_dim_row_count/stride`, `idx_mapping_ptr` parameters, and `num_loops` for DS conv copy. | Must match upstream kernel contract. | ##### 11. `tests/e2e/conftest.py` / `tests/ut/spec_decode/test_speculators_vwn_eagle3.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | HunyuanVL placeholder version gate; remove unused `maybe_calc_kv_scales` mock | [vllm #49691](vllm-project/vllm#49691), #49389. | Version gate and dead-mock removal. | Adapt to upstream contract changes. | ##### 12. `vllm_ascend/patch/__init__.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Document FusedMoE → FusedMoEFactory rename and new patch_vision entry | — | Update patch registry documentation. | Keep the patch manifest in sync with reality. | #### Compatibility and review notes - Version gates use `vllm_version_is("0.26.0")` exclusively; no `hasattr` fallbacks beyond the explicitly justified `clear_buffer` guard (where the upstream change is a method removal, not a rename). - The `FusedMoE` → `FusedMoEFactory` rename is applied consistently across all call sites: `deepseek_v4.py`, `minimax_m3.py`, `fused_moe.py`, `routed_experts.py`, and `patch_fused_moe.py`. - The `layer.py` `Q/K/V_SCALE_CONSTANT` removal is a dead-code cleanup: the `DSAAttention` class initialized these tensors from `envs` module-level constants (which still exist), but never used them in `forward()`. ### Does this PR introduce _any_ user-facing change? No. This is a compatibility update; no new Ascend-specific public API is introduced. ### How was this patch tested? CI on the branch. See Buildkite workflow run for detailed results. - vLLM version: v0.26.0 - vLLM main: vllm-project/vllm@0351e9a --------- Signed-off-by: liaoqidan <1107297340@qq.com>
shiqiangA
pushed a commit
to shiqiangA/vllm-ascend
that referenced
this pull request
Aug 20, 2026
### What this PR does / why we need it? #### Upgrade baseline - Update the verified vLLM main anchor from [`2e09247c2d7b6b97d13af6e71a85bf8d1271deb6`](vllm-project/vllm@2e09247) to [`58d3918e3ea0a544ffedadad2ba84559e9c51d8f`](vllm-project/vllm@58d3918). The full upstream range is available in this [comparison](vllm-project/vllm@2e09247...58d3918). - Preserve the vLLM `0.26.0` compatibility lane while adapting the main lane to the new upstream contracts. Version gates use `vllm_version_is("0.26.0")` and are limited to real contract differences. - The changes are organized in the same order as the changed files in this PR. Each item identifies the upstream change, the downstream adaptation, and why the adaptation is required. #### Changes by file ##### 1. `.github/vllm-main-verified.commit` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Update anchor to `58d3918e` | Upgrade window [0351e9aa...58d3918e](vllm-project/vllm@0351e9a...58d3918). | Set anchor. | Source of truth for main2main workflow. | ##### 2. `vllm_ascend/patch/platform/patch_fused_moe.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate FusedMoE → FusedMoEFactory rename | [vllm #44941](vllm-project/vllm#44941) renamed `FusedMoE` to `FusedMoEFactory`. | On main, capture and patch `FusedMoEFactory`; on v0.26.0, also patch legacy `FusedMoE`. | Both lanes need the Ascend runner patch at the correct binding. | ##### 3. `vllm_ascend/models/deepseek_v4.py` / `vllm_ascend/models/minimax_m3/minimax_m3.py` / `vllm_ascend/ops/fused_moe/fused_moe.py` / `vllm_ascend/ops/fused_moe/routed_experts.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Import and use `FusedMoEFactory`; remove dead `FusedMoE` re-export | [vllm #44941](vllm-project/vllm#44941). | Replace `FusedMoE` with `FusedMoEFactory`. | Old symbol no longer exists on main. Remove stale `FusedMoE` re-export from `fused_moe.py` and dead reference in `routed_experts.py` comment. | ##### 4. `tests/ut/models/test_deepseek_v4_moe.py` / `tests/ut/models/minimax_m3/test_minimax_m3.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Update monkeypatch target to `FusedMoEFactory` | [vllm #44941](vllm-project/vllm#44941). | `"FusedMoE"` → `"FusedMoEFactory"`. | Must match the symbol imported by models. | ##### 5. `vllm_ascend/worker/model_runner_v1.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate `calculate_kv_scales` removal | [vllm #49389](vllm-project/vllm#49389) removed runtime KV-scale calculation. | Add `vllm_version_is("0.26.0")` guard. | Ascend MRV1 still supports it on v0.26.0; attribute absent on main. | | Version-gate `clear_buffer()` removal | [vllm #50721](vllm-project/vllm#50721) removed `clear_buffer()` from `RoutedExpertsCapturer`. | Wrap in `vllm_version_is("0.26.0")` guard. | On main, each routed layer overwrites current step's token rows. | ##### 6. `vllm_ascend/models/layer/attention/layer.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Remove dead `Q/K/V_SCALE_CONSTANT` references | [vllm #49389](vllm-project/vllm#49389) removed env var registrations. Module-level constants still exist. | Remove unused `q_range`/`k_range`/`v_range` initializations and dead `import envs`. | Dead-code cleanup; `DSAAttention.forward()` never used these attributes. | ##### 7. `tests/ut/patch/platform/test_deepseek_v4_thinking.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate reasoning effort expectations | [vllm #50580](vllm-project/vllm#50580) maps `low`/`minimal`/`medium` → `low`. | `vllm_version_is("0.26.0")` guard. | v0.26.0 keeps old mapping; main uses new. | ##### 8. `tests/e2e/pull_request/one_card/spec_decode/test_extract_hidden_states.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Enable `chunked_prefill` for hybrid model | [vllm #50991](vllm-project/vllm#50991) enabled prefix cache by default for Mamba/hybrid models. | `False` → `True`. | Hybrid model now requires chunked prefill. | ##### 9. `vllm_ascend/patch/platform/patch_vision.py` (new) + `vllm_ascend/patch/platform/__init__.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Patch `FusedInputNorm.forward` eps=0.0 → eps=1e-5 | [vllm #50411](vllm-project/vllm#50411) added `FusedInputNorm` with `F.batch_norm(eps=0.0)`. | Monkey-patch forward to use `eps=1e-5`; guarded with `contextlib.suppress(ImportError)`. | Upstream PyTorch 2.13.0 allows eps >= 0 for inference; vllm-ascend PyTorch 2.10.0 requires eps > 0 always. Release wheels lack `FusedInputNorm`. Remove this patch once bundled PyTorch >= 2.13.0. | ##### 10. `vllm_ascend/ops/triton/mamba/postprocess.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Add kernel signature parameters | [vllm #50432](vllm-project/vllm#50432) changed signature. | Add `CONV_STATE_DIM_FIRST`, `HAS_IDX_MAPPING`, `PRECOMPUTED_NEW_COMPUTED`, `state_dim_row_count/stride`, `idx_mapping_ptr` parameters, and `num_loops` for DS conv copy. | Must match upstream kernel contract. | ##### 11. `tests/e2e/conftest.py` / `tests/ut/spec_decode/test_speculators_vwn_eagle3.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | HunyuanVL placeholder version gate; remove unused `maybe_calc_kv_scales` mock | [vllm #49691](vllm-project/vllm#49691), #49389. | Version gate and dead-mock removal. | Adapt to upstream contract changes. | ##### 12. `vllm_ascend/patch/__init__.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Document FusedMoE → FusedMoEFactory rename and new patch_vision entry | — | Update patch registry documentation. | Keep the patch manifest in sync with reality. | #### Compatibility and review notes - Version gates use `vllm_version_is("0.26.0")` exclusively; no `hasattr` fallbacks beyond the explicitly justified `clear_buffer` guard (where the upstream change is a method removal, not a rename). - The `FusedMoE` → `FusedMoEFactory` rename is applied consistently across all call sites: `deepseek_v4.py`, `minimax_m3.py`, `fused_moe.py`, `routed_experts.py`, and `patch_fused_moe.py`. - The `layer.py` `Q/K/V_SCALE_CONSTANT` removal is a dead-code cleanup: the `DSAAttention` class initialized these tensors from `envs` module-level constants (which still exist), but never used them in `forward()`. ### Does this PR introduce _any_ user-facing change? No. This is a compatibility update; no new Ascend-specific public API is introduced. ### How was this patch tested? CI on the branch. See Buildkite workflow run for detailed results. - vLLM version: v0.26.0 - vLLM main: vllm-project/vllm@0351e9a --------- Signed-off-by: liaoqidan <1107297340@qq.com>
frankie-ys
pushed a commit
to Csrayz/vllm-ascend
that referenced
this pull request
Aug 26, 2026
### What this PR does / why we need it? #### Upgrade baseline - Update the verified vLLM main anchor from [`2e09247c2d7b6b97d13af6e71a85bf8d1271deb6`](vllm-project/vllm@2e09247) to [`58d3918e3ea0a544ffedadad2ba84559e9c51d8f`](vllm-project/vllm@58d3918). The full upstream range is available in this [comparison](vllm-project/vllm@2e09247...58d3918). - Preserve the vLLM `0.26.0` compatibility lane while adapting the main lane to the new upstream contracts. Version gates use `vllm_version_is("0.26.0")` and are limited to real contract differences. - The changes are organized in the same order as the changed files in this PR. Each item identifies the upstream change, the downstream adaptation, and why the adaptation is required. #### Changes by file ##### 1. `.github/vllm-main-verified.commit` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Update anchor to `58d3918e` | Upgrade window [0351e9aa...58d3918e](vllm-project/vllm@0351e9a...58d3918). | Set anchor. | Source of truth for main2main workflow. | ##### 2. `vllm_ascend/patch/platform/patch_fused_moe.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate FusedMoE → FusedMoEFactory rename | [vllm #44941](vllm-project/vllm#44941) renamed `FusedMoE` to `FusedMoEFactory`. | On main, capture and patch `FusedMoEFactory`; on v0.26.0, also patch legacy `FusedMoE`. | Both lanes need the Ascend runner patch at the correct binding. | ##### 3. `vllm_ascend/models/deepseek_v4.py` / `vllm_ascend/models/minimax_m3/minimax_m3.py` / `vllm_ascend/ops/fused_moe/fused_moe.py` / `vllm_ascend/ops/fused_moe/routed_experts.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Import and use `FusedMoEFactory`; remove dead `FusedMoE` re-export | [vllm #44941](vllm-project/vllm#44941). | Replace `FusedMoE` with `FusedMoEFactory`. | Old symbol no longer exists on main. Remove stale `FusedMoE` re-export from `fused_moe.py` and dead reference in `routed_experts.py` comment. | ##### 4. `tests/ut/models/test_deepseek_v4_moe.py` / `tests/ut/models/minimax_m3/test_minimax_m3.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Update monkeypatch target to `FusedMoEFactory` | [vllm #44941](vllm-project/vllm#44941). | `"FusedMoE"` → `"FusedMoEFactory"`. | Must match the symbol imported by models. | ##### 5. `vllm_ascend/worker/model_runner_v1.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate `calculate_kv_scales` removal | [vllm #49389](vllm-project/vllm#49389) removed runtime KV-scale calculation. | Add `vllm_version_is("0.26.0")` guard. | Ascend MRV1 still supports it on v0.26.0; attribute absent on main. | | Version-gate `clear_buffer()` removal | [vllm #50721](vllm-project/vllm#50721) removed `clear_buffer()` from `RoutedExpertsCapturer`. | Wrap in `vllm_version_is("0.26.0")` guard. | On main, each routed layer overwrites current step's token rows. | ##### 6. `vllm_ascend/models/layer/attention/layer.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Remove dead `Q/K/V_SCALE_CONSTANT` references | [vllm #49389](vllm-project/vllm#49389) removed env var registrations. Module-level constants still exist. | Remove unused `q_range`/`k_range`/`v_range` initializations and dead `import envs`. | Dead-code cleanup; `DSAAttention.forward()` never used these attributes. | ##### 7. `tests/ut/patch/platform/test_deepseek_v4_thinking.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Version-gate reasoning effort expectations | [vllm #50580](vllm-project/vllm#50580) maps `low`/`minimal`/`medium` → `low`. | `vllm_version_is("0.26.0")` guard. | v0.26.0 keeps old mapping; main uses new. | ##### 8. `tests/e2e/pull_request/one_card/spec_decode/test_extract_hidden_states.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Enable `chunked_prefill` for hybrid model | [vllm #50991](vllm-project/vllm#50991) enabled prefix cache by default for Mamba/hybrid models. | `False` → `True`. | Hybrid model now requires chunked prefill. | ##### 9. `vllm_ascend/patch/platform/patch_vision.py` (new) + `vllm_ascend/patch/platform/__init__.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Patch `FusedInputNorm.forward` eps=0.0 → eps=1e-5 | [vllm #50411](vllm-project/vllm#50411) added `FusedInputNorm` with `F.batch_norm(eps=0.0)`. | Monkey-patch forward to use `eps=1e-5`; guarded with `contextlib.suppress(ImportError)`. | Upstream PyTorch 2.13.0 allows eps >= 0 for inference; vllm-ascend PyTorch 2.10.0 requires eps > 0 always. Release wheels lack `FusedInputNorm`. Remove this patch once bundled PyTorch >= 2.13.0. | ##### 10. `vllm_ascend/ops/triton/mamba/postprocess.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Add kernel signature parameters | [vllm #50432](vllm-project/vllm#50432) changed signature. | Add `CONV_STATE_DIM_FIRST`, `HAS_IDX_MAPPING`, `PRECOMPUTED_NEW_COMPUTED`, `state_dim_row_count/stride`, `idx_mapping_ptr` parameters, and `num_loops` for DS conv copy. | Must match upstream kernel contract. | ##### 11. `tests/e2e/conftest.py` / `tests/ut/spec_decode/test_speculators_vwn_eagle3.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | HunyuanVL placeholder version gate; remove unused `maybe_calc_kv_scales` mock | [vllm #49691](vllm-project/vllm#49691), #49389. | Version gate and dead-mock removal. | Adapt to upstream contract changes. | ##### 12. `vllm_ascend/patch/__init__.py` | Change | Upstream change | Downstream adaptation | Why | |---|---|---|---| | Document FusedMoE → FusedMoEFactory rename and new patch_vision entry | — | Update patch registry documentation. | Keep the patch manifest in sync with reality. | #### Compatibility and review notes - Version gates use `vllm_version_is("0.26.0")` exclusively; no `hasattr` fallbacks beyond the explicitly justified `clear_buffer` guard (where the upstream change is a method removal, not a rename). - The `FusedMoE` → `FusedMoEFactory` rename is applied consistently across all call sites: `deepseek_v4.py`, `minimax_m3.py`, `fused_moe.py`, `routed_experts.py`, and `patch_fused_moe.py`. - The `layer.py` `Q/K/V_SCALE_CONSTANT` removal is a dead-code cleanup: the `DSAAttention` class initialized these tensors from `envs` module-level constants (which still exist), but never used them in `forward()`. ### Does this PR introduce _any_ user-facing change? No. This is a compatibility update; no new Ascend-specific public API is introduced. ### How was this patch tested? CI on the branch. See Buildkite workflow run for detailed results. - vLLM version: v0.26.0 - vLLM main: vllm-project/vllm@0351e9a --------- Signed-off-by: liaoqidan <1107297340@qq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
#41184 deleted the
FusedMoEclass and replaced it with a function that constructed aMoERunnerand related MoE classes. This PR renamesFusedMoE->FusedMoEFactoryto better reflect the purpose of the function.Test Plan
CI/Lint
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.