Skip to content

Fix NextN weight loading by sharing the unified-loader mappings - #33478

Merged
yueming-yuan merged 1 commit into
sglang-milesfrom
yueming/nextn-unified-loader-attrs
Aug 4, 2026
Merged

yueming-yuan merged 1 commit into
sglang-milesfrom
yueming/nextn-unified-loader-attrs

Conversation

@yueming-yuan

@yueming-yuan yueming-yuan commented Aug 4, 2026 •

Copy link
Copy Markdown
Collaborator

Loading EAGLE draft weights for a DeepSeek-V3 NextN model through the unified weight loading API crashes on missing attributes.

DeepseekV3ForCausalLMNextN.__init__ calls nn.Module.__init__ and rebuilds the parts of the parent init it needs, so it never reaches where DeepseekV3ForCausalLM.__init__ sets fuse_qkv_a_proj, stacked_params_mapping and expert_params_mapping — the three attributes the unified loader reads off the model.

Rather than copy that block into the subclass, this extracts it into DeepseekV3ForCausalLM.init_unified_loader_mappings() and calls it from both __init__s, right after determine_num_fused_shared_experts() (which expert_params_mapping depends on). A subclass that builds its own __init__ now has one call to make instead of a block to keep in sync.

fuse_qkv_a_proj is still assigned early in the parent, before packed_modules_mapping needs it; the helper recomputes the same value from config, so the parent behaviour is unchanged.

Repro: GLM-5.2 744B with --speculative-algorithm EAGLE on the glm_moe_dsa (DeepSeek-V3 lineage) path.

🤖 Generated with Claude Code


CI States

Latest PR Test (Base): ❌ Run #30877585251
Latest PR Test (Extra): ❌ Run #30877585103

DeepseekV3ForCausalLMNextN builds its own __init__ and never reaches the
parent's assignments of fuse_qkv_a_proj, stacked_params_mapping and
expert_params_mapping, so loading EAGLE draft weights through the unified
API dies on the missing attributes.

Extract them into DeepseekV3ForCausalLM.init_unified_loader_mappings() and
call it from both, so a subclass with its own __init__ has one call to
make instead of a block to keep in sync.
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@yueming-yuan
yueming-yuan merged commit 31ad05a into sglang-miles Aug 4, 2026
72 of 82 checks passed
@yueming-yuan
yueming-yuan deleted the yueming/nextn-unified-loader-attrs branch August 4, 2026 04:28
Kh4L pushed a commit to Kh4L/sglang that referenced this pull request Sep 8, 2026
Zhichenzzz pushed a commit that referenced this pull request Sep 19, 2026
…gs (#33478)

(cherry picked from commit e8fecd1)
(cherry picked from commit 263beea)
HJSang pushed a commit to HJSang/sglang that referenced this pull request Sep 21, 2026
ziang-and pushed a commit to zianglih/sglang that referenced this pull request Oct 2, 2026
…ism (sgl-project#21278, sgl-project#22663, sgl-project#33478)

Expose rank topology for P2P weight mapping and create replicas using scoped runtime-context overrides. Restore the previous configuration on exit, including failed context entry.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: yueming-yuan <yym022502@gmail.com>
nv-hwoo pushed a commit to nv-hwoo/sglang that referenced this pull request Oct 20, 2026
…ism (sgl-project#21278, sgl-project#22663, sgl-project#33478)

Expose rank topology for P2P weight mapping and create replicas using scoped runtime-context overrides. Restore the previous configuration on exit, including failed context entry.

Co-authored-by: yueming-yuan <yym022502@gmail.com>
Rockdu pushed a commit that referenced this pull request Oct 30, 2026
…ism (#21278, #22663, #33478)

Expose rank topology for P2P weight mapping and create replicas using scoped runtime-context overrides. Restore the previous configuration on exit, including failed context entry.

Co-authored-by: yueming-yuan <yym022502@gmail.com>
Rockdu pushed a commit that referenced this pull request Oct 30, 2026
…ism (#21278, #22663, #33478)

Expose rank topology for P2P weight mapping and create replicas using scoped runtime-context overrides. Restore the previous configuration on exit, including failed context entry.

Co-authored-by: yueming-yuan <yym022502@gmail.com>
Rockdu pushed a commit that referenced this pull request Oct 31, 2026
…ism (#21278, #22663, #33478)

Expose rank topology for P2P weight mapping and create replicas using scoped runtime-context overrides. Restore the previous configuration on exit, including failed context entry.

Co-authored-by: yueming-yuan <yym022502@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant