Skip to content

[DSV4.1][PD][3/N] Avoid host synchronization in DeepSeek V4.1 CP prefill - #40335

Closed
BJWang-ant wants to merge 1 commit into
sgl-project:mainfrom
BJWang-ant:main-v4.1-cp-bcg-opt-sync
Closed

[DSV4.1][PD][3/N] Avoid host synchronization in DeepSeek V4.1 CP prefill#40335
BJWang-ant wants to merge 1 commit into
sgl-project:mainfrom
BJWang-ant:main-v4.1-cp-bcg-opt-sync

Conversation

@BJWang-ant

@BJWang-ant BJWang-ant commented Sep 19, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Contributor

issue：#39700
base pr：#40333

Summary

image image The DeepSeek V4.1 CP prefill indexer introduces host synchronization when uploading CPU metadata and indexing the request-to-token table with a CUDA scalar. These waits prevent the CPU from submitting subsequent work asynchronously.
  • Upload query lengths and key offsets through the existing pinned-memory, non-blocking transfer helper.
  • Keep request indices as one-dimensional CUDA tensors to avoid implicit device-to-host scalar reads.
  • Add a regression test that rejects CUDA scalar reads during mixed-length prefill, including requests with zero query tokens.

perf

image

CI States

Latest PR Test (Base): ❌ Run #35436933948
Latest PR Test (Extra): ❌ Run #35436933799
Latest PR Test (AMD ROCm 10): ❌ Run #35436933971

@github-actions github-actions Bot added Multi-modal multi-modal language model deepseek labels Sep 19, 2026
@BJWang-ant BJWang-ant mentioned this pull request Sep 19, 2026
Open
2 tasks done
@BJWang-ant
BJWang-ant force-pushed the main-v4.1-cp-bcg-opt-sync branch from b79508d to 5105960 Compare September 19, 2026 10:16
@Oasis-Git Oasis-Git mentioned this pull request Oct 2, 2026
Open
17 of 56 tasks
@BJWang-ant

BJWang-ant commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

#41125 same work

@BJWang-ant BJWang-ant closed this Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@merrymercy merrymercy Awaiting requested review from merrymercy merrymercy is a code owner
@Ying1123 Ying1123 Awaiting requested review from Ying1123
@hnyls2002 hnyls2002 Awaiting requested review from hnyls2002
@Fridge003 Fridge003 Awaiting requested review from Fridge003 Fridge003 is a code owner
@ispobock ispobock Awaiting requested review from ispobock ispobock is a code owner
@xiezhq-hermann xiezhq-hermann Awaiting requested review from xiezhq-hermann
@hanming-lu hanming-lu Awaiting requested review from hanming-lu
@yizhang2077 yizhang2077 Awaiting requested review from yizhang2077
@hzh0425 hzh0425 Awaiting requested review from hzh0425
@alphabetc1 alphabetc1 Awaiting requested review from alphabetc1
@huangtingwei9988 huangtingwei9988 Awaiting requested review from huangtingwei9988
@Qiaolin-Yu Qiaolin-Yu Awaiting requested review from Qiaolin-Yu Qiaolin-Yu is a code owner
@hebiao064 hebiao064 Awaiting requested review from hebiao064 hebiao064 is a code owner
@HaiShaw HaiShaw Awaiting requested review from HaiShaw HaiShaw is a code owner
@ch-wan ch-wan Awaiting requested review from ch-wan
@BBuf BBuf Awaiting requested review from BBuf
@Edwardf0t1 Edwardf0t1 Awaiting requested review from Edwardf0t1
@ByronHsu ByronHsu Awaiting requested review from ByronHsu
@ShangmingCai ShangmingCai Awaiting requested review from ShangmingCai
@Duyi-Wang Duyi-Wang Awaiting requested review from Duyi-Wang
@sogalin sogalin Awaiting requested review from sogalin

Assignees

No one assigned

Labels

deepseek Multi-modal multi-modal language model

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@BJWang-ant