Skip to content

[Bugfix] Wrong Answer in PD disaggregation [P w/o SPS | D w/ SPS] #17212

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ShangmingCai merged 3 commits into from
Jan 18, 2026
+11 −0
Merged

[Bugfix] Wrong Answer in PD disaggregation [P w/o SPS | D w/ SPS] #17212

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d2a81b9
[Bugfix] Fix incorrect responses when using PD disaggregation: P w/o …
Ch3ngY1 Jan 16, 2026
551f2e4
Update python/sglang/srt/disaggregation/common/conn.py
ShangmingCai Jan 16, 2026
49682ca
fix lint
ShangmingCai Jan 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions python/sglang/srt/disaggregation/common/conn.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,17 @@ def get_mha_kv_ptrs_with_pp(
if num_kv_layers == dst_num_total_layers:
dst_k_ptrs = dst_kv_ptrs[:dst_num_total_layers]
dst_v_ptrs = dst_kv_ptrs[dst_num_total_layers:]
elif (
num_kv_layers < dst_num_total_layers
and dst_num_total_layers % num_kv_layers != 0
):
# Case: Decode has more layers than Prefill (e.g., Decode has draft model KV while Prefill is deployed without speculative decoding)
# To prevent empty Value Cache, which leads to wrong response
# dst_kv_ptrs layout: [K_main..., V_main..., draft_K..., draft_V...]
dst_k_ptrs = dst_kv_ptrs[start_layer:end_layer]
dst_v_ptrs = dst_kv_ptrs[
num_kv_layers + start_layer : num_kv_layers + end_layer
]
else:
# Decode pp size should be equal to prefill pp size or 1
dst_k_ptrs = dst_kv_ptrs[start_layer:end_layer]
Expand Down
Loading