Repository navigation
[npu]add chunk gdn kernel and unify ssm state layout for ascend gdn backend - #36187
Merged
Merged
Conversation
Signed-off-by: zhaozx-cn <zhaozx2116@163.com>
Signed-off-by: zhaozx-cn <zhaozx2116@163.com>
zhaozx-cn
requested review from
Ying1123,
alphabetc1,
hanming-lu,
hnyls2002,
huangtingwei9988,
hzh0425,
iforgetmyname,
ispobock,
merrymercy,
ping1jing2,
whybeyoung,
xiezhq-hermann and
yizhang2077
as code owners
August 24, 2026 12:56
AndyLi429
reviewed
Aug 25, 2026
AndyLi429
reviewed
Aug 25, 2026
AndyLi429
reviewed
Aug 25, 2026
zhaozx-cn
commented
Aug 25, 2026
zhaozx-cn
commented
Aug 25, 2026
sglang-npu-bot
approved these changes
Sep 17, 2026
Collaborator
|
/tag-and-rerun-ci |
Contributor
Author
|
/rerun-failed-ci |
Contributor
Author
|
/rerun-failed-ci |
Contributor
Author
|
/rerun-failed-ci |
3 similar comments
Contributor
Author
|
/rerun-failed-ci |
Contributor
Author
|
/rerun-failed-ci |
Contributor
Author
|
/rerun-failed-ci |
Contributor
Author
|
/rerun-failed-ci |
2 similar comments
Contributor
Author
|
/rerun-failed-ci |
Contributor
Author
|
/rerun-failed-ci |
Contributor
Author
|
/rerun-failed-ci |
3 similar comments
Contributor
Author
|
/rerun-failed-ci |
Contributor
Author
|
/rerun-failed-ci |
Contributor
Author
|
/rerun-failed-ci |
4 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
chunk gated delta rule is now ready. with this operator adopted, the SSM states in both the prefill and MTP branches become (dv, dk), which allows us to unify the SSM state layout. this unification eliminates unnecessary transpose operations, further improving prefill performance.
Modifications
replace triton fla kernel with chunk gated delta rule.
unify ssm states layout for prefill/decode/mtp.
Accuracy Tests
qwen 3.6 35b ceval
official: 90.0
this pr: 90.1
Speed Tests and Profiling
qwen 3.6 35b
4k in 1k out NEXTN 3
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): ❌ Run #35504083021
Latest PR Test (Extra): ❌ Run #35504082792
Latest PR Test (AMD ROCm 10): ⏳ Run #35504082956