[Model] Support GLM-5.3 Flash NVFP4 loading - #38621
Merged
YAMY1234 merged 3 commits intoSep 9, 2026
Merged
Conversation
nvpohanh
commented
Sep 9, 2026
nvpohanh
commented
Sep 9, 2026
nvpohanh
marked this pull request as ready for review
September 9, 2026 05:49
Collaborator
Author
|
/rerun-tests test/registered/e2e/models/test_glm53_flash_b200.py |
Contributor
|
Results for 🚀 |
YAMY1234
approved these changes
Sep 9, 2026
Collaborator
|
/rerun-test test/registered/unit/models/test_glm5_next_modelopt.py |
Contributor
|
Results for 🚀 |
mmangkad
requested changes
Sep 9, 2026
mmangkad
approved these changes
Sep 9, 2026
Collaborator
|
Wait for #38621 (comment) to pass |
Collaborator
|
CI is flaky, related test passed: https://github.com/sgl-project/sglang/actions/runs/34319931588 |
pllimax
added a commit
to pllimax/sglang
that referenced
this pull request
Sep 10, 2026
* origin/main: (27 commits) [Simulator] Give the OFFLINE/BLOCKING comparison tolerances real headroom (sgl-project#38732) [Config] msgspec.Struct for the config tier (sgl-project#38753) [AMD] ci: move the miles nightlies from rocm700 to rocm10 (sgl-project#37495) [Config] One writer for the declaration stash; no exception to the write seal (sgl-project#38752) docker(xpu): drop redundant setvars.sh from torch_memory_saver RUN (sgl-project#38665) [XPU][Fix] Pack device-pointer tables as uint64 to avoid 64-bit address overflow (sgl-project#35051) [CI] Temporarily disable GB300 tests (sgl-project#38770) [diffusion] feat: spill large tensors over shared memory like numpy arrays (sgl-project#38656) [diffusion] refactor: refactor utility ownership and document helper placement (sgl-project#38699) [NPU]Support GLM5.2 and FP8 DSA&Indexer kvcache for 950 (sgl-project#38250) [CI] Answer unrecognized slash commands instead of skipping silently (sgl-project#38736) [AMD] Parallelize aiter spec-decode KV index building over token blocks (sgl-project#37659) [DSv4] Integrate TRT-LLM DSv4 Attention for SM100/103 (sgl-project#30805) Add Opt-In for GLM-5.3 Flash breakable prefill CUDA graphs (sgl-project#38522) [CI] Install helion 1.4.0 for the KDA Helion kernel tests (sgl-project#38688) [Rust] Gate health on startup warmup completion (sgl-project#37994) [HiCache] Replace skip_lock_node_ids with a segment lock protocol (sgl-project#36848) feat: add optimized Domino rollout to DFlash V2 (sgl-project#36899) [CI] Add /run-full-ci and /run-extra-ci slash commands (sgl-project#38734) [Model] Support GLM-5.3 Flash NVFP4 loading (sgl-project#38621) ...
This was referenced Sep 10, 2026
Leoyzen
pushed a commit
to Leoyzen/sglang
that referenced
this pull request
Sep 10, 2026
Co-authored-by: YAMY <74099316+YAMY1234@users.noreply.github.com> (cherry picked from commit 96d91ef)
mqhc2020
pushed a commit
to mqhc2020/sglang
that referenced
this pull request
Sep 15, 2026
Co-authored-by: YAMY <74099316+YAMY1234@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[by Codex]
Motivation
nvidia/GLM-5.3-Flash-NVFP4is a mixed-precision ModelOpt checkpoint: attention and shared-expert modules remain unquantized while routed experts use NVFP4.Two GLM-5-specific mismatches currently prevent the checkpoint from loading:
model.language_model.*, while the corresponding SGLang modules are undermodel.*. The weight loader already performs this rename, but precision selection did not.This PR is intentionally narrower than #36761 and #37322. It does not change generic packed-module matching or redesign NextN quantization. Current
mainalready keeps the GLM-5 NextN layer unquantized.Modifications
Accuracy Tests
Validated an unchanged snapshot of
nvidia/GLM-5.3-Flash-NVFP4on 4x NVIDIA GB300 usinglmsysorg/sglang:dev-cu13and the requested launch configuration. The server became healthy without editing checkpoint metadata and without passing--disable-shared-experts-fusion:Basic deterministic GSM8K subset:
Local regression tests:
The tested set includes
test_modelopt_nvfp4.py,test_glm5_next_dflash_capture.py,test_shared_experts_fusion_gates.py, and the new GLM-5 ModelOpt tests.Speed Tests and Profiling
Not applicable. This only changes checkpoint interpretation and prevents an invalid fusion combination; uniformly quantized configurations retain the existing fusion path.
Checklist
pre-commit run --all-files --show-diff-on-failuresuccessfully.CI States
Latest PR Test (Base): ❌ Run #34386744816
Latest PR Test (Extra): ❌ Run #34386744469
Latest PR Test (AMD ROCm 10): ❌ Run #34386744775