Skip to content

[bugfix]Enable dispatch_ffn_combine feature for qwen3.5#7066

Merged
MengqingCao merged 14 commits intovllm-project:mainfrom
asunxiao:main
Mar 17, 2026
Merged

[bugfix]Enable dispatch_ffn_combine feature for qwen3.5#7066
MengqingCao merged 14 commits intovllm-project:mainfrom
asunxiao:main

Conversation

@asunxiao
Copy link
Copy Markdown
Contributor

@asunxiao asunxiao commented Mar 9, 2026

What this PR does / why we need it?

Qwen3.5 Moe supports enabling the dispatch_ffn_combine fusion operator.

Fix problem: In the w8a8 quantization scene, Qwen3.5 model's config.json lacks the quantize field. The previous logic strictly relied on quant_type == "w8a8_dynamic" to enable VLLM_ASCEND_ENABLE_FUSED_MC2. This caused the dispatch_ffn_combine fusion operator to fail to activate even when the environment variable was set.

Enable dispatch_ffn_combine fusion operator for BF16 scenarios.

Does this PR introduce any user-facing change?

How was this patch tested?

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 9, 2026

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enables the dispatch_ffn_combine fusion operator, specifically for Qwen3.5 Mixture of Experts (MoE) models. The change removes a previous dependency on a specific quantization type (w8a8_dynamic) for the fused_mc2_enable flag, thereby broadening the applicability of this performance optimization. This aims to improve the efficiency and compatibility of MoE models on Ascend hardware.

Highlights

  • Feature Enablement: Enabled the dispatch_ffn_combine fusion operator for Qwen3.5 MoE models.
  • Quantization Restriction Removal: Removed the w8a8_dynamic quantization type restriction for the fused_mc2_enable condition, allowing broader use of the dispatch_ffn_combine feature.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • vllm_ascend/ascend_forward_context.py
    • Removed the quant_type variable definition.
    • Updated the fused_mc2_enable condition to no longer check for a specific quantization type.
Activity
  • No specific activity (comments, reviews, progress) has been recorded for this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request aims to enable the dispatch_ffn_combine fusion operator for Qwen3.5 MoE models by removing the W8A8 quantization check. While this enables the feature for more models, it introduces a critical issue. The change allows non-quantized models to use a communication path (FusedMC2CommImpl) that strictly requires quantization scales, which will lead to a runtime crash. My review includes a critical comment detailing the issue and suggesting a path forward to correctly and safely implement this feature.

Comment thread vllm_ascend/ascend_forward_context.py
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 9, 2026

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 9, 2026

This pull request has conflicts, please resolve those before we can evaluate the pull request.

@asunxiao asunxiao closed this Mar 9, 2026
@asunxiao asunxiao reopened this Mar 9, 2026
@asunxiao asunxiao changed the title Enable dispatch_ffn_combine feature [bugfix]Enable dispatch_ffn_combine feature for qwen3.5 Mar 10, 2026
Comment thread vllm_ascend/ops/fused_moe/moe_comm_method.py Outdated
Comment thread vllm_ascend/ascend_forward_context.py
Signed-off-by: asunxiao <asunxiao@qq.com>
Signed-off-by: asunxiao <asunxiao@qq.com>
Signed-off-by: asunxiao <asunxiao@qq.com>
Signed-off-by: asunxiao <asunxiao@qq.com>
Signed-off-by: asunxiao <asunxiao@qq.com>
Signed-off-by: asunxiao <asunxiao@qq.com>
@github-actions
Copy link
Copy Markdown
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Signed-off-by: asunxiao <asunxiao@qq.com>
Signed-off-by: asunxiao <asunxiao@qq.com>
Signed-off-by: asunxiao <asunxiao@qq.com>
Signed-off-by: asunxiao <asunxiao@qq.com>
@github-actions
Copy link
Copy Markdown
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Signed-off-by: asunxiao <asunxiao@qq.com>
@MengqingCao
Copy link
Copy Markdown
Collaborator

Could you add a test case for this scenario?

Comment thread vllm_ascend/ops/fused_moe/fused_moe.py
Comment thread vllm_ascend/ops/fused_moe/fused_moe.py
Comment thread vllm_ascend/ops/fused_moe/fused_moe.py
Signed-off-by: asunxiao <asunxiao@qq.com>
Signed-off-by: asunxiao <asunxiao@qq.com>
Signed-off-by: asunxiao <asunxiao@qq.com>
Copy link
Copy Markdown
Collaborator

@MengqingCao MengqingCao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now, thanks!

@MengqingCao MengqingCao added ready read for review ready-for-test start test by label for PR labels Mar 17, 2026
@MengqingCao MengqingCao merged commit a370dfa into vllm-project:main Mar 17, 2026
75 of 79 checks passed
845473182 pushed a commit to 845473182/vllm-ascend that referenced this pull request Mar 18, 2026
…scend into qwen3next_graph

* 'qwen3next_graph' of https://github.com/845473182/vllm-ascend: (62 commits)
  [doc] Refresh the documentation for DeepSeek-V3.2 (vllm-project#7403)
  [bugfix][accuracy] Fix ds indexer accuracy problem caused by k rope (vllm-project#7341)
  [P/D] LayerwiseConnector supports the virtual push functionality on node D. (vllm-project#7361)
  [CI] Add PAT_TOKEN when checkout (vllm-project#7400)
  [main2main] upgrade vllm to 0308 (vllm-project#7213)
  [CI] add scheduled stale issue management (vllm-project#7354)
  [CI] expand issue labeler rules for feature/model triage (vllm-project#7356)
  [Bugfix] Assertion error when decode prefix cache fully hits (vllm-project#7236)
  [doc] Refresh the documentation for GLM-4.7 (vllm-project#7292)
  [BugFix]A2 MOE method&& layerwise MTP bugfix && Mamba gdn_metadata bugfix (vllm-project#7364)
  [doc] Upload doc for qwen3.5-27B and qwen3.5-397B-A17B on Ascend (vllm-project#7313)
  [bugfix]Enable dispatch_ffn_combine feature for qwen3.5 (vllm-project#7066)
  [bugfix] fix unzip file path for fia operator (vllm-project#7367)
  [Perf] Optimize bias handling in AscendRMSNorm (vllm-project#7226)
  [eagle3][pcp] fix bug for eagle3 and cp enable (vllm-project#7309)
  [Bugfix] fix TransposeKvCacheByBlock op error report in plog (vllm-project#7235)
  [Feature]Supports DSv3.1 PD separation and C8 quantization (vllm-project#7222)
  [main][bugfix] Fixed the problem that eagle3 will crash in FULL_DECODE_ONLY (vllm-project#7290)
  [xlite][Bugfix] Support mrope and deepstack features in xlite backend (vllm-project#7295)
  [model_runner_v2]optimize the performance of the _topk_log_softmax_kernel (vllm-project#7221)
  ...
liuchenbing2026 pushed a commit to liuchenbing2026/vllm-ascend that referenced this pull request Mar 19, 2026
…#7066)

Cherry-pick from upstream main a370dfa.
Enable dispatch_ffn_combine fusion operator for BF16 and w8a8 scenarios
by removing strict quantization type dependencies.
starmountain1997 pushed a commit to starmountain1997/vllm-ascend that referenced this pull request Mar 25, 2026
…#7066)

### What this PR does / why we need it?
Qwen3.5 Moe supports enabling the dispatch_ffn_combine fusion operator.

Fix problem: In the w8a8 quantization scene, Qwen3.5 model's config.json
lacks the quantize field. The previous logic strictly relied on
quant_type == "w8a8_dynamic" to enable VLLM_ASCEND_ENABLE_FUSED_MC2.
This caused the dispatch_ffn_combine fusion operator to fail to activate
even when the environment variable was set.
Enable dispatch_ffn_combine fusion operator for BF16 scenarios.

- vLLM version: v0.16.0
- vLLM main:
vllm-project/vllm@4034c3d
---------
Signed-off-by: asunxiao <asunxiao@qq.com>
lihaokun-2026 pushed a commit to lihaokun-2026/vllm-ascend that referenced this pull request Mar 29, 2026
…#7066)

### What this PR does / why we need it?
Qwen3.5 Moe supports enabling the dispatch_ffn_combine fusion operator.

Fix problem: In the w8a8 quantization scene, Qwen3.5 model's config.json
lacks the quantize field. The previous logic strictly relied on
quant_type == "w8a8_dynamic" to enable VLLM_ASCEND_ENABLE_FUSED_MC2.
This caused the dispatch_ffn_combine fusion operator to fail to activate
even when the environment variable was set.
Enable dispatch_ffn_combine fusion operator for BF16 scenarios.

- vLLM version: v0.16.0
- vLLM main:
vllm-project/vllm@4034c3d
---------
Signed-off-by: asunxiao <asunxiao@qq.com>
chenchuw886 pushed a commit to chenchuw886/vllm-ascend that referenced this pull request Apr 1, 2026
…#7066)

### What this PR does / why we need it?
Qwen3.5 Moe supports enabling the dispatch_ffn_combine fusion operator.

Fix problem: In the w8a8 quantization scene, Qwen3.5 model's config.json
lacks the quantize field. The previous logic strictly relied on
quant_type == "w8a8_dynamic" to enable VLLM_ASCEND_ENABLE_FUSED_MC2.
This caused the dispatch_ffn_combine fusion operator to fail to activate
even when the environment variable was set.
Enable dispatch_ffn_combine fusion operator for BF16 scenarios.

- vLLM version: v0.16.0
- vLLM main:
vllm-project/vllm@4034c3d
---------
Signed-off-by: asunxiao <asunxiao@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:core ready read for review ready-for-test start test by label for PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants