Skip to content

Revert deepseek_weight_loader.py changes from #25141 - #28561

Merged
yushengsu-thu merged 1 commit into
sgl-project:sglang-milesfrom
nanjiangwill:revert-deepseek-loader-25141
Jun 30, 2026
Merged

Revert deepseek_weight_loader.py changes from #25141#28561
yushengsu-thu merged 1 commit into
sgl-project:sglang-milesfrom
nanjiangwill:revert-deepseek-loader-25141

Conversation

@nanjiangwill

@nanjiangwill nanjiangwill commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • The deepseek_weight_loader.py changes in support kimi 2.5/6 lora (logprob diff exist) #25141 (persistent `cached_a_proj` + unconditional `.detach().clone()`) were a workaround for `q_a_proj` and `kv_a_proj_with_mqa` landing in different chunks during chunked weight updates. The fusion to `fused_qkv_a_proj_with_mqa` only fires when both halves are visible in the same `load_weights` call, so without persistence, a split chunk produced partially-fused weights.
  • The proper fix is on the sender side: pair the two halves before chunking so they always arrive in the same `load_weights` call. Done in radixark/miles#1361, mirroring the approach slime adopted in THUDM/slime#1753.
  • With sender-side pairing in place, `cached_a_proj` only needs to live for one `load_weights` call, matching the original upstream behavior. This PR reverts the loader back to that form.

Why revert on `sglang-miles` and not main

Test plan


CI States

Latest PR Test (Base): ❌ Run #28424125993
Latest PR Test (Extra): ❌ Run #28424125889

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

The persistent cached_a_proj and unconditional .detach().clone()
were a workaround for q_a_proj / kv_a_proj_with_mqa landing in
different chunks during chunked weight updates. The proper fix is
on the sender side: pair the two halves before chunking so they
always arrive in the same load_weights call. That fix is in
radixark/miles#1361 (mirrors THUDM/slime#1532).

With sender-side pairing in place, the cached_a_proj only ever
needs to live for one load_weights call, matching the original
upstream behavior.
@nanjiangwill
nanjiangwill force-pushed the revert-deepseek-loader-25141 branch from 2830e3f to 18f96d9 Compare June 30, 2026 06:09
@yushengsu-thu
yushengsu-thu merged commit c855d1b into sgl-project:sglang-miles Jun 30, 2026
37 of 42 checks passed
@nanjiangwill
nanjiangwill deleted the revert-deepseek-loader-25141 branch June 30, 2026 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants