[Feat][SP] Suport SP for VL MoE models

[realliujiaxu]

What this PR does / why we need it?

2nd PR for #5712, extend SP to VL MoE models.

Does this PR introduce any user-facing change?

remove sp_threshold in additional config and reuse sp_min_token_num from vLLM.

How was this patch tested?

• Model: Qwen3-VL-30B-A3B,
• TP4 DP2
• 100 reqs
• max concurrency 1

Seq length	Mean TTFT (ms) main	Mean TTFT (ms) this PR
4k	429.40	323.3
16k	1297.01	911.74

• vLLM version: v0.16.0
• vLLM main: vllm-project/vllm@4034c3d

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the sequence parallelism (SP) compilation pass architecture to enhance modularity and maintainability. It introduces a two-stage pass system, separating AllReduce and AllGather pattern applications, and integrates a new pass for eliminating redundant graph operations. Concurrently, the documentation for sequence parallelism has been comprehensively updated to reflect these architectural changes, providing clearer explanations and pattern details for users and developers. These changes streamline the SP implementation without altering user-facing APIs or behavior, focusing on internal code quality and clarity.

Highlights

• Pass Architecture Refactor: The sequence parallelism (SP) pass architecture has been refactored into two distinct passes: SequenceParallelismPass and SequenceParallelismAllgatherEpPass. The SequenceParallelismPass now focuses on AllReduce-based patterns, while the new SequenceParallelismAllgatherEpPass handles AllGather-related patterns, improving modularity and clarity.
• Pattern Reorganization and Naming Convention: AllGather patterns, including those for middle and last layers and Qwen3-VL, have been moved into the new sequence_parallelism_moe.py file. The AllGatherChunkNoOpCleanupPass logic has been merged into SequenceParallelismAllgatherEpPass as the AllGatherChunkNoOpPattern. Additionally, the 'Ascend' prefix has been removed from pass and pattern names for better generalization.
• No-Op Elimination Pass: A new NoOpEliminationPass has been introduced to remove redundant view-like operations (e.g., view, reshape) from the computation graph, optimizing the graph before applying SP patterns.
• Enhanced Documentation: The sequence_parallelism.md user guide has been significantly updated. It now includes a detailed 'Pass Design' section with match/replacement tables for all patterns, an explanation for Qwen3-VL's special handling, and a dedicated section for SequenceParallelismAllgatherEpPass with an overview and diagram.
• Conditional FlashComm V1 and SP Enablement: Logic has been updated in custom operations and platform configuration to conditionally enable FlashComm V1 and handle SP-related communication based on the new pass-based SP enablement flag (enable_sp_by_pass), ensuring correct behavior during graph compilation.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• docs/source/user_guide/feature_guide/sequence_parallelism.md
  • Added a 'Pass Design' section detailing the new SequenceParallelismPass and SequenceParallelismAllgatherEpPass.
  • Included match and replacement tables for various SP patterns.
  • Provided an explanation for the special handling required by Qwen3-VL models.
  • Added an overview and diagram for the AllGather EP computation graph and cleanup.
• vllm_ascend/compilation/graph_fusion_pass_manager.py
  • Updated the configure method to append SequenceParallelismPass and SequenceParallelismAllgatherEpPass when sequence parallelism is enabled.
• vllm_ascend/compilation/passes/allgather_chunk_noop_pass.py
  • Added a new pass file AllGatherChunkNoOpCleanupPass which defines a pattern to fold redundant all_gather and chunk operations, though this pattern is now integrated into SequenceParallelismAllgatherEpPass.
• vllm_ascend/compilation/passes/noop_elimination.py
  • Added a new pass NoOpEliminationPass to remove no-op view/reshape nodes from the computation graph.
• vllm_ascend/compilation/passes/sequence_parallelism.py
  • Renamed AscendMiddleAllReduceRMSNormPattern to MiddleAllReduceRMSNormPattern.
  • Renamed AscendLastAllReduceRMSNormPattern to LastAllReduceRMSNormPattern.
  • Renamed AscendQwen3VLMiddleAllReduceRMSNormPattern to Qwen3VLMiddleAllReduceRMSNormPattern.
  • Renamed AscendSequenceParallelismPass to SequenceParallelismPass.
  • Imported and integrated NoOpEliminationPass to run before applying SP patterns.
• vllm_ascend/compilation/passes/sequence_parallelism_moe.py
  • Added a new file to define SequenceParallelismAllgatherEpPass.
  • Defined MiddleLayerAllgatherAddRMSNormPattern to postpone all_gather for middle layers.
  • Defined LastLayerAllgatherRMSNormPattern for last layers with all_gather + RMSNorm fusion.
  • Defined Qwen3VLMiddleLayerAllgatherAddRMSNormPattern for Qwen3-VL specific all_gather paths.
  • Defined AllGatherChunkNoOpPattern to fold all_gather + sequence_parallel_chunk_impl into an identity operation.
• vllm_ascend/ops/fused_moe/prepare_finalize.py
  • Modified prepare and finalize methods to check enable_sp_by_pass() in addition to enable_sp() for conditional logic.
• vllm_ascend/ops/register_custom_ops.py
  • Updated _maybe_all_gather_and_maybe_unpad_impl to consider enable_sp_by_pass() for FlashComm V1 enablement and added a TODO for unpadding.
  • Updated _maybe_pad_and_reduce_impl to consider enable_sp_by_pass() for FlashComm V1 enablement and added a TODO for padding.
  • Updated _maybe_pad_and_reduce_fake to consider enable_sp_by_pass().
• vllm_ascend/patch/worker/patch_distributed.py
  • Added an all_gather method to the GroupCoordinatorPatch class.
• vllm_ascend/platform.py
  • Modified check_and_update_config to conditionally set parallel_config.all2all_backend based on whether SP is enabled via passes.
• vllm_ascend/utils.py
  • Added a global variable _ENABLE_SP_BY_PASS for caching the SP enablement status.
  • Refactored enable_sp_by_pass function for proper initialization, caching, and handling of None for vllm_config.
• vllm_ascend/worker/worker.py
  • Modified the profile method signature to include an optional profile_prefix parameter.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request refactors the sequence parallelism (SP) pass architecture by splitting it into two passes, SequenceParallelismPass and SequenceParallelismAllgatherEpPass, and reorganizes the associated patterns. The documentation for SP is also significantly improved. The changes are mostly well-structured, but there are a few critical issues. I've found some incomplete logic marked with TODO comments in custom ops that could lead to bugs, an unused parameter in a method, and an obsolete file that should be removed. Please address these points to ensure the correctness and quality of the code.

I am having trouble creating individual review comments. Click here to see my feedback.

vllm_ascend/ops/register_custom_ops.py (56-57)

The TODO: do unpad comment indicates that the unpadding logic is missing when enable_sp_by_pass() is true. This will cause the function to return a padded tensor, which can lead to shape mismatches and incorrect results in downstream operations. This should be implemented to ensure correctness.

vllm_ascend/ops/register_custom_ops.py (93-94)

The TODO: do pad comment indicates that the padding logic is missing when enable_sp_by_pass() is true. The function performs reduce_scatter without padding the input tensor x. This can lead to incorrect behavior or errors if the downstream operations expect a padded tensor. The padding logic should be implemented here.

vllm_ascend/compilation/passes/allgather_chunk_noop_pass.py (1-40)

This file seems to be obsolete. The PR description states that AllGatherChunkNoOpCleanupPass is merged into SequenceParallelismAllgatherEpPass as AllGatherChunkNoOpPattern. The new file vllm_ascend/compilation/passes/sequence_parallelism_moe.py already contains AllGatherChunkNoOpPattern. This file allgather_chunk_noop_pass.py defining AllGatherChunkNoOpCleanupPass appears to be a leftover and should be removed to avoid confusion and dead code.

vllm_ascend/worker/worker.py (514)

The newly added parameter profile_prefix is not used within the profile method. It should either be used or removed to avoid confusion and maintain clean code.

    def profile(self, is_start: bool = True):

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[wxsIcey]

Not familiar with the details of the SP, but from pass perspective, approve.