Skip to content

Cherry pick weight_checker fp8 dequant fix and non-persistent buffer skip from #21494 - #24532

Merged
fzyzcjy merged 1 commit into
sgl-project:mainfrom
fzyzcjy:cherry_pick_weight_checker/0
May 6, 2026
Merged

Cherry pick weight_checker fp8 dequant fix and non-persistent buffer skip from #21494#24532
fzyzcjy merged 1 commit into
sgl-project:mainfrom
fzyzcjy:cherry_pick_weight_checker/0

Conversation

@fzyzcjy

@fzyzcjy fzyzcjy commented May 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Cherry-picks the weight_checker.py change from #21494 (originally landed on the sglang-miles branch) onto main.

Original author: @yym022502 (Yueming Yuan). I'm only the courier.

Changes

  • Skip cos_sin_cache / freqs_cis buffers in _reset_tensors (non-persistent, get recomputed after weight load).
  • Add weight_scale_inv names to skip_compare_names so fp8 scale tensors aren't double-compared.
  • Make inverse_transform_scale_ue8m0 conditional on w_s.dtype == torch.int32 (UE8M0 packed format / Blackwell DeepGEMM only); other paths now feed w_s directly to block_quant_dequant.

PR chain

First of 3 PRs, all touching weight_checker.py only, cherry-picking weight_checker-related changes from sglang-miles to main:

  1. This PR — cherry-pick of [sglang-miles] fix weight checker #21494
  2. Cherry-pick of weight_checker portion of P2P Weight Update features for miles  #21278
  3. Cherry-pick of weight_checker portion of [Hotfix] final fixes for P2P Transfer  #22663

The next two PRs each cherry-pick only the weight_checker.py slice of their source PR (the originals also touched many unrelated files). Please merge in order.

Test plan

  • CI green

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@fzyzcjy
fzyzcjy force-pushed the cherry_pick_weight_checker/0 branch from 839c3ac to 35df793 Compare May 6, 2026 13:09
@fzyzcjy fzyzcjy changed the title Cherry-pick weight_checker fix from #21494 to main Cherry pick weight_checker fp8 dequant fix and non-persistent buffer skip from #21494 May 6, 2026
@fzyzcjy
fzyzcjy merged commit 864f963 into sgl-project:main May 6, 2026
55 of 64 checks passed
LLThomas pushed a commit to LLThomas/sglang that referenced this pull request May 8, 2026
Chronostasys pushed a commit to MindLab-Research/sglang that referenced this pull request Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants