[CI/UT][Graph] Add ut for torchair graph mode#1103
Merged
Conversation
MengqingCao
force-pushed
the
torchair
branch
7 times, most recently
from
1cd7fd4 to
590b412
Compare
| num_heads = query.size(1) | ||
| block_size = kv_cache.size(1) | ||
| latent_kv_dim = kv_cache.size(3) - rope_dim | ||
| block_size = kv_cache[0].size(1) |
Collaborator
There was a problem hiding this comment.
Torchair will get into this code path?
Collaborator
There was a problem hiding this comment.
yes, with chunked prefill enabled.
but this is another bug. we should fix it later. @MengqingCao will add the test with ascend scheudler enabled.
Collaborator
Author
There was a problem hiding this comment.
I have removed this fix now because there still exists oom issue on chunked prefill senario.
This pr will focus ut on ascend scheudler with chunked prefill disabled. cc @Yikun
Collaborator
Author
There was a problem hiding this comment.
This pr is ready for review now. plz take a look, thx!
github-actions
Bot
added
documentation
* use vllm-ascend/DeepSeek-V3 * use spawn * update cleanup dist env and mem * update torchair config * disable eager * enable refresh * use random weight * some fixes Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: MengqingCao <cmq0113@163.com>
Signed-off-by: Mengqing Cao <cmq0113@163.com>
Yikun
approved these changes
Jun 14, 2025
Signed-off-by: Mengqing Cao <cmq0113@163.com>
wangxiyuan
approved these changes
Jun 14, 2025
Member
|
Let's merge this to protect DeepSeek graph mode on V1 (with |
Yikun
reviewed
Jun 14, 2025
| "ascend_scheduler_config": { | ||
| "enabled": True, | ||
| }, | ||
| "refresh": True, |
Member
There was a problem hiding this comment.
last question: why need refresh here?
Member
There was a problem hiding this comment.
I merged this first feel free to open new PR to address this if needed
Collaborator
Author
There was a problem hiding this comment.
refresh is used here because an initial setup of llm is already done in UTs, it need a refresh of additional config. And we don't need this in a normal usage
83 tasks
momo609
pushed a commit
to momo609/vllm-ascend
that referenced
this pull request
Jun 17, 2025
### What this PR does / why we need it? Add ut for torchair graph mode on DeepSeekV3 ### How was this patch tested? CI passed with new added test. --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Mengqing Cao <cmq0113@163.com>
momo609
pushed a commit
to momo609/vllm-ascend
that referenced
this pull request
Jun 17, 2025
### What this PR does / why we need it? Add ut for torchair graph mode on DeepSeekV3 ### How was this patch tested? CI passed with new added test. --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Mengqing Cao <cmq0113@163.com> Signed-off-by: wangxiaoxin (A) <wangxiaoxin7@huawei.com>
momo609
pushed a commit
to momo609/vllm-ascend
that referenced
this pull request
Jun 17, 2025
### What this PR does / why we need it? Add ut for torchair graph mode on DeepSeekV3 ### How was this patch tested? CI passed with new added test. --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Mengqing Cao <cmq0113@163.com> Signed-off-by: wangxiaoxin (A) <wangxiaoxin7@huawei.com>
shiyuan680
pushed a commit
to raindaywhu/vllm-ascend
that referenced
this pull request
Jul 7, 2025
Add ut for torchair graph mode on DeepSeekV3 CI passed with new added test. --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Mengqing Cao <cmq0113@163.com>
chopper0126
pushed a commit
to chopper0126/vllm-ascend
that referenced
this pull request
Oct 16, 2025
### What this PR does / why we need it? Add ut for torchair graph mode on DeepSeekV3 ### How was this patch tested? CI passed with new added test. --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Mengqing Cao <cmq0113@163.com>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
### What this PR does / why we need it? Add ut for torchair graph mode on DeepSeekV3 ### How was this patch tested? CI passed with new added test. --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Mengqing Cao <cmq0113@163.com>
yangzhe-2026
pushed a commit
to yangzhe-2026/vllm-ascend
that referenced
this pull request
May 6, 2026
### What this PR does / why we need it? Add ut for torchair graph mode on DeepSeekV3 ### How was this patch tested? CI passed with new added test. --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Mengqing Cao <cmq0113@163.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it?
Add ut for torchair graph mode on DeepSeekV3
How was this patch tested?
CI passed with new added test.