Skip to content

[csrc][bugfix] Add compile-time Ascend950/910_95 compatibility for custom ops between CANN8.5 and 9.0#6936

Merged
wangxiyuan merged 2 commits intovllm-project:mainfrom
linfeng-yuan:a3_kernel_compilation_with_cann90
Mar 3, 2026
Merged

[csrc][bugfix] Add compile-time Ascend950/910_95 compatibility for custom ops between CANN8.5 and 9.0#6936
wangxiyuan merged 2 commits intovllm-project:mainfrom
linfeng-yuan:a3_kernel_compilation_with_cann90

Conversation

@linfeng-yuan
Copy link
Copy Markdown
Collaborator

@linfeng-yuan linfeng-yuan commented Mar 2, 2026

What this PR does / why we need it?

Remove hardcoded ASCEND910_95 usage in csrc custom-op host/tiling code and
select the SoC target at CMake configure time.

  • Probe CANN headers with check_cxx_source_compiles:
    prefer platform_ascendc::SocVersion::ASCEND950, fallback to ASCEND910_95.
  • Export the selected enum/config string via shared compile definitions (VLLM_ASCEND_950_SOC_ENUM / VLLM_ASCEND_950_SOC_CONFIG).
  • Apply the shared macros to affected paths (moe_gating_top_k, add_rms_norm_bias) to avoid per-file hardcoding.
  • Keep behavior unchanged; this is an internal build-compatibility fix for CANN 8.5 and 9.x.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Signed-off-by: linfeng-yuan <1102311262@qq.com>
@linfeng-yuan linfeng-yuan requested a review from zzzzwwjj as a code owner March 2, 2026 16:50
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses compatibility issues when compiling against different versions of the CANN toolkit (specifically 8.5 and 9.0). It implements a robust mechanism to dynamically detect the specific Ascend SoC enum (either ASCEND950 or ASCEND910_95) available in the installed CANN headers. This detected version is then used consistently throughout the C++ codebase, ensuring that operators and tiling configurations are correctly compiled regardless of the CANN version, thereby preventing build failures.

Highlights

  • Dynamic SoC Version Detection: Introduced dynamic detection of Ascend SoC version (ASCEND950 or ASCEND910_95) at compile time using CMake's CheckCXXSourceCompiles module, ensuring compatibility with different CANN versions.
  • Consistent SoC Configuration: Propagated the detected SoC version and configuration as C++ compile definitions (VLLM_ASCEND_950_SOC_ENUM, VLLM_ASCEND_950_SOC_CONFIG) to ensure consistent usage across the codebase.
  • Operator and Tiling Logic Updates: Updated operator definitions and tiling logic in various files to utilize the dynamically determined SoC version instead of hardcoded values, improving flexibility and preventing build failures.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • csrc/add_rms_norm_bias/op_host/add_rms_norm_bias_tiling.h
    • Updated the socVersion field in AddRmsNormBiasCompileInfo to use the dynamically defined VLLM_ASCEND_950_SOC_ENUM.
  • csrc/cmake/config.cmake
    • Included the CheckCXXSourceCompiles CMake module.
    • Implemented logic to detect the presence of ASCEND950 or ASCEND910_95 SoC versions within CANN headers.
    • Defined VLLM_ASCEND_950_SOC_ENUM and VLLM_ASCEND_950_SOC_CONFIG CMake variables based on the detected SoC version.
  • csrc/cmake/intf_pub.cmake
    • Added VLLM_ASCEND_950_SOC_ENUM and VLLM_ASCEND_950_SOC_CONFIG as C++ compile definitions for the intf_pub target.
  • csrc/moe_gating_top_k/op_host/moe_gating_top_k_def.cpp
    • Modified the AICore().AddConfig call to use the dynamically determined VLLM_ASCEND_950_SOC_CONFIG.
  • csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling_arch35.cpp
    • Renamed the tiling key constant from MOE_GATING_TOP_K_REGBASE_TILING_KEY to MOE_GATING_TOP_K_ASCEND_950_TILING_KEY.
    • Updated the IsCapable method to check against platform_ascendc::SocVersion::VLLM_ASCEND_950_SOC_ENUM.
    • Updated the GetTilingKey method to return the newly named tiling key constant.
  • csrc/moe_init_routing_custom/op_host/moe_init_routing_custom_tiling.cpp
    • Updated a comment for the REGISTER_TILING_TEMPLATE to clarify its role as a fallback for non-Ascend950 SoCs.
Activity
  • No specific activity or review comments were provided in the pull request description.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 2, 2026

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

@linfeng-yuan linfeng-yuan changed the title fix compatibility of kernel compilation between CANN8.5 and 9.0 [ops] fix compatibility of kernel compilation between CANN8.5 and 9.0 Mar 2, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a compile-time mechanism to handle different SoC enum values between CANN versions, improving build compatibility. The changes are logically sound, but I've found a critical issue in the runtime check that could break compatibility between different compile-time and run-time CANN versions. I've provided suggestions to fix this in csrc/cmake/intf_pub.cmake and csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling_arch35.cpp.

Additionally, the pull request description is currently empty. Per the repository's style guide, I've drafted a suggested title and summary:

Suggested PR Title:

[Build][BugFix] Fix kernel compilation compatibility for different CANN versions

Suggested PR Summary:

### What this PR does / why we need it?
This PR addresses a compilation issue when building vLLM kernels against different versions of Huawei's CANN toolkit (e.g., 8.5 and 9.0). These versions use different enum values for the Ascend 950 series SoC (`ASCEND910_95` vs `ASCEND950`).

To resolve this, the PR introduces a compile-time check in CMake to detect which enum value is supported by the CANN headers. The result is then passed to the C++ code via preprocessor definitions (`VLLM_ASCEND_950_SOC_ENUM` and `VLLM_ASCEND_950_SOC_CONFIG`). This allows the code to use the correct SoC identifier, ensuring compilation compatibility across different CANN versions.

### Does this PR introduce _any_ user-facing change?
No. This is a build-time compatibility fix and does not change any user-facing APIs or behavior.

### How was this patch tested?
The changes were tested by compiling the kernels against different CANN versions to ensure compatibility. CI tests should be run to confirm correctness and prevent runtime regressions.

Comment thread csrc/cmake/intf_pub.cmake
Comment thread csrc/moe_gating_top_k/op_host/moe_gating_top_k_tiling_arch35.cpp
@linfeng-yuan linfeng-yuan changed the title [ops] fix compatibility of kernel compilation between CANN8.5 and 9.0 [csrc] Add compile-time Ascend950/910_95 compatibility for custom ops between CANN8.5 and 9.0 Mar 2, 2026
@linfeng-yuan
Copy link
Copy Markdown
Collaborator Author

This solution cannot cover the cross-version case of “build with CANN 9.0 + run with CANN 8.5,” and that scenario itself is high-risk.

Why:

The current selection logic is decided once at build time (config.cmake probes headers and fixes macros); it is not re-evaluated at runtime.
If built with 9.0, it will be fixed to the ASCEND950/ascend950 path, which may not correctly match 910_95 semantics when running with 8.5.
More importantly, binary compatibility is a concern: custom ops/tiling/op host usually depend on the CANN headers and libraries used at build time, and ABI/behavior compatibility is not guaranteed across major runtime versions.

@linfeng-yuan linfeng-yuan added ready read for review ready-for-test start test by label for PR labels Mar 2, 2026
Signed-off-by: linfeng-yuan <1102311262@qq.com>
@linfeng-yuan linfeng-yuan changed the title [csrc] Add compile-time Ascend950/910_95 compatibility for custom ops between CANN8.5 and 9.0 [csrc][bugfix] Add compile-time Ascend950/910_95 compatibility for custom ops between CANN8.5 and 9.0 Mar 3, 2026
@wangxiyuan wangxiyuan merged commit cb893bc into vllm-project:main Mar 3, 2026
27 checks passed
845473182 pushed a commit to 845473182/vllm-ascend that referenced this pull request Mar 5, 2026
…to qwen3next_graph

* 'main' of https://github.com/vllm-project/vllm-ascend: (40 commits)
  [Feature] Add docs of batch invariance and make some extra operators patch (vllm-project#6910)
  [bugfix]Qwen2.5VL accurate question (vllm-project#6975)
  [CI] Add DeepSeek-V3.2 large EP nightly ci (vllm-project#6378)
  [Ops][BugFix] Fix RoPE shape mismatch for mtp models with flashcomm v1 enabled (vllm-project#6939)
  [bugfix]fix file not found error in nightly of single-node (vllm-project#6976)
  [Bugfix] Fix the acceptance rates dorp issue when applying eagle3 to QuaRot model (vllm-project#6914)
  [CI] Enable auto upgrade e2e estimated time for auto-partition suites (vllm-project#6840)
  [Doc][Misc] Fix msprobe_guide.md documentation issues (vllm-project#6965)
  [Nightly][Refactor]Migrate nightly single-node model tests from `.py` to `.yaml` (vllm-project#6503)
  [BugFix] Improve GDN layer detection for multimodal models (vllm-project#6941)
  [feat]ds3.2 pcp support mtp and chunkprefill (vllm-project#6917)
  [CPU binding] Implement global CPU slicing and improve IRQ binding for Ascend NPUs (vllm-project#6945)
  [Triton] Centralize Ascend extension op dispatch in triton_utils (vllm-project#6937)
  [csrc][bugfix] Add compile-time Ascend950/910_95 compatibility for custom ops between CANN8.5 and 9.0 (vllm-project#6936)
  [300I][Bugfix] fix unquant model weight nd2nz error (vllm-project#6851)
  [doc] fix supported_models (vllm-project#6930)
  [CI] nightly test timeout (vllm-project#6912)
  [CI] Upgrade CANN to 8.5.1 (vllm-project#6897)
  [Model]Add Qwen3-Omni quantization Ascend NPU adaptation and optimization (vllm-project#6828)
  [P/D][v0.16.0]Adapt to RecomputeScheduler in vLLM 0.16.0 (vllm-project#6898)
  ...
LCAIZJ pushed a commit to LCAIZJ/vllm-ascend that referenced this pull request Mar 7, 2026
…stom ops between CANN8.5 and 9.0 (vllm-project#6936)

### What this PR does / why we need it?
Remove hardcoded ASCEND910_95 usage in csrc custom-op host/tiling code
and
select the SoC target at CMake configure time.

- Probe CANN headers with check_cxx_source_compiles:
prefer platform_ascendc::SocVersion::ASCEND950, fallback to
ASCEND910_95.
- Export the selected enum/config string via shared compile definitions
(VLLM_ASCEND_950_SOC_ENUM / VLLM_ASCEND_950_SOC_CONFIG).
- Apply the shared macros to affected paths (moe_gating_top_k,
add_rms_norm_bias) to avoid per-file hardcoding.
- Keep behavior unchanged; this is an internal build-compatibility fix
for CANN 8.5 and 9.x.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?

- vLLM version: v0.16.0
- vLLM main:
vllm-project/vllm@15d76f7

---------

Signed-off-by: linfeng-yuan <1102311262@qq.com>
zjchenn pushed a commit to zjchenn/vllm-ascend that referenced this pull request Mar 10, 2026
…stom ops between CANN8.5 and 9.0 (vllm-project#6936)

### What this PR does / why we need it?
Remove hardcoded ASCEND910_95 usage in csrc custom-op host/tiling code
and
select the SoC target at CMake configure time.

- Probe CANN headers with check_cxx_source_compiles:
prefer platform_ascendc::SocVersion::ASCEND950, fallback to
ASCEND910_95.
- Export the selected enum/config string via shared compile definitions
(VLLM_ASCEND_950_SOC_ENUM / VLLM_ASCEND_950_SOC_CONFIG).
- Apply the shared macros to affected paths (moe_gating_top_k,
add_rms_norm_bias) to avoid per-file hardcoding.
- Keep behavior unchanged; this is an internal build-compatibility fix
for CANN 8.5 and 9.x.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?

- vLLM version: v0.16.0
- vLLM main:
vllm-project/vllm@15d76f7

---------

Signed-off-by: linfeng-yuan <1102311262@qq.com>
weijinqian0 added a commit that referenced this pull request Mar 16, 2026
… compatibility for custom ops between CANN8.5 and 9.0 (#7116)

cherry-pick from #6936
to fix pip install error

### What this PR does / why we need it?

Remove hardcoded ASCEND910_95 usage in csrc custom-op host/tiling code
and
select the SoC target at CMake configure time.

- Probe CANN headers with check_cxx_source_compiles: prefer
platform_ascendc::SocVersion::ASCEND950, fallback to ASCEND910_95.
- Export the selected enum/config string via shared compile definitions
(VLLM_ASCEND_950_SOC_ENUM / VLLM_ASCEND_950_SOC_CONFIG).
- Apply the shared macros to affected paths (moe_gating_top_k,
add_rms_norm_bias) to avoid per-file hardcoding.
- Keep behavior unchanged; this is an internal build-compatibility fix
for CANN 8.5 and 9.x.

### Does this PR introduce _any_ user-facing change? No.

### How was this patch tested?

Signed-off-by: linfeng-yuan <1102311262@qq.com>
Co-authored-by: linfeng-yuan <1102311262@qq.com>
Co-authored-by: weijinqian0 <1184188277@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready read for review ready-for-test start test by label for PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants