Repository navigation
Fix NextN weight loading by sharing the unified-loader mappings - #33478
Merged
Merged
Conversation
DeepseekV3ForCausalLMNextN builds its own __init__ and never reaches the parent's assignments of fuse_qkv_a_proj, stacked_params_mapping and expert_params_mapping, so loading EAGLE draft weights through the unified API dies on the missing attributes. Extract them into DeepseekV3ForCausalLM.init_unified_loader_mappings() and call it from both, so a subclass with its own __init__ has one call to make instead of a block to keep in sync.
yueming-yuan
requested review from
Fridge003,
ch-wan,
fzyzcjy,
ispobock and
merrymercy
as code owners
August 4, 2026 04:24
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
yueming-yuan
added a commit
that referenced
this pull request
Aug 25, 2026
Kh4L
pushed a commit
to Kh4L/sglang
that referenced
this pull request
Sep 8, 2026
…gs (sgl-project#33478) (cherry picked from commit e8fecd1)
8 of 33 tasks
HJSang
pushed a commit
to HJSang/sglang
that referenced
this pull request
Sep 21, 2026
…gs (sgl-project#33478) (cherry picked from commit e8fecd1)
ziang-and
pushed a commit
to zianglih/sglang
that referenced
this pull request
Oct 2, 2026
…ism (sgl-project#21278, sgl-project#22663, sgl-project#33478) Expose rank topology for P2P weight mapping and create replicas using scoped runtime-context overrides. Restore the previous configuration on exit, including failed context entry. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: yueming-yuan <yym022502@gmail.com>
nv-hwoo
pushed a commit
to nv-hwoo/sglang
that referenced
this pull request
Oct 20, 2026
…ism (sgl-project#21278, sgl-project#22663, sgl-project#33478) Expose rank topology for P2P weight mapping and create replicas using scoped runtime-context overrides. Restore the previous configuration on exit, including failed context entry. Co-authored-by: yueming-yuan <yym022502@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Loading EAGLE draft weights for a DeepSeek-V3 NextN model through the unified weight loading API crashes on missing attributes.
DeepseekV3ForCausalLMNextN.__init__callsnn.Module.__init__and rebuilds the parts of the parent init it needs, so it never reaches whereDeepseekV3ForCausalLM.__init__setsfuse_qkv_a_proj,stacked_params_mappingandexpert_params_mapping— the three attributes the unified loader reads off the model.Rather than copy that block into the subclass, this extracts it into
DeepseekV3ForCausalLM.init_unified_loader_mappings()and calls it from both__init__s, right afterdetermine_num_fused_shared_experts()(whichexpert_params_mappingdepends on). A subclass that builds its own__init__now has one call to make instead of a block to keep in sync.fuse_qkv_a_projis still assigned early in the parent, beforepacked_modules_mappingneeds it; the helper recomputes the same value from config, so the parent behaviour is unchanged.Repro: GLM-5.2 744B with
--speculative-algorithm EAGLEon the glm_moe_dsa (DeepSeek-V3 lineage) path.🤖 Generated with Claude Code
CI States
Latest PR Test (Base): ❌ Run #30877585251
Latest PR Test (Extra): ❌ Run #30877585103