moe: the shared-experts-fusion decision is a per-runner value the loader installs

[ch-wan]

What

A draft's construction used to rewrite the process config record. Two writers, two shapes:

1. The shared-experts-fusion decision becomes a per-runner value the loader installs. declare_load_time_override wrote the fusion decision to the target's bags — and an MTP/nextn draft IS a DeepSeek/GLM/Qwen3.5/MiniMax model, so a draft whose checkpoint differs from the target's (quantization) corrupted the target's record. This is ancestral behavior: the pre-refactor code wrote the shared ServerArgs object.

The decision is per runner, and it is now made in one place — the loader's single model-instantiation point, before any layer exists:

_initialize_model()                      # the only place a model class is built
  install_shared_experts_fusion_decision(model_class, hf_config, quant_config)
  model_class(**kwargs)                  # the layers read the installed value

• flags.moe.disable_shared_experts_fusion carries it, in the runner_backend / speculative_runner_backend shape: initialize_moe_config seeds both leaves with the user's intent, and draft_model_build_scope — entered by every draft build — routes the draft's answer onto the speculative leaf as well, so a flags dump afterwards shows both runners' decisions. The accessor refuses forward-time reads: a forward reads what its build baked in (num_fused_shared_experts on the layer), and during a draft's build the ACTIVE leaf holds the draft's value.

• Each family that can auto-disable exposes its conditions as shared_experts_fusion_disable_reason(hf_config, quant_config), answering from the config and quantization it is handed — no instance, no layers. That is where the conditions already lived, and the loader is the last point that still knows both the checkpoint's quantization and whether this runner is a draft. determine_num_fused_shared_experts shrinks to reading the installed decision, keeping each family's asserts and logs. A family without a gate (qwen2_moe) follows the user's intent.

• The architecture a class fuses for becomes a class attribute (fused_shared_experts_architecture) instead of an argument threaded through the call: the NextN drafts and GLM's DSA variant override it, which also removes the super() hop in GlmMoeDsaForCausalLM.

• A wrapper answers with the config it hands its nested body. The loader asks the class it instantiates, which for several archs is a wrapper around a DeepSeek/Qwen3.5 body — and those conditions used to be evaluated during that nested construction. Each wrapper therefore delegates with the config (and quantization) its constructor passes down, derived by a function the constructor shares so the two cannot drift apart:

  entry class	nested body	what it hands over
  KimiK25ForConditionalGeneration	DeepseekV3ForCausalLM	text_config; encoder-only → None
  KimiVLForConditionalGeneration	DeepseekV2ForCausalLM	text_config deep-copied with architectures rewritten, exactly as the constructor does
  DotsVLMForCausalLM	DeepseekV2ForCausalLM	language_config
  DeepseekVL2ForCausalLM	DeepseekV2ForCausalLM	language_config, no quantization (the construction passes none); non-MLA builds the dense model → None
  DeepseekOCRForCausalLM	DeepseekV2/V3ForCausalLM	text_config, MoE branches only (OCR2 / non-MLA → None)
  PixtralForConditionalGeneration	MistralLarge3ForCausalLM	text_config when it is deepseek_v3; the GQA branch builds dense Mistral → None
  MiniCPMV, MiniCPMV4_6ForConditionalGeneration	Qwen3_5ForCausalLM	text_config
  Qwen3_5ForCausalLMMTP	Qwen3_5ForCausalLM	text_config + the MTP module's own quantization normalization (_mtp_quant_config, shared with the constructor)
  qwen3_5_text.Qwen3_5{,Moe}ForCausalLM	body_cls	its config verbatim, following the same attribute the constructor uses

  qwen3_5 needed particular care twice: its auto-disable is ROCm-only and reads the text config while the loader instantiates the VL entry classes, and qwen3_5_text.py defines its own classes with the same names as the multimodal ones — the registry resolves text-only checkpoints to those, so attaching the gate to the multimodal classes alone would have silently stopped the ROCm auto-disable.

The bag leaf is the user's pristine intent again, consistent with the kv_cache_dtype ruling, and with the last caller gone (the device-driven sm80 dtype fallback, identical for every runner in the process, records directly through get_context().override) declare_load_time_override is deleted.

draft_model_build_scope deliberately does not swap runner_backend: that is speculative_moe_backend_context's job and has to bracket the draft's whole lifecycle, which dflash/dspark/standalone do not do — a construction-only swap would build and execute the draft under different backends.

2. A draft weight update recorded the draft's model_path/load_format as the process's (today the scheduler feeds both runners the same path, so the write was same-value redundant); the record is target-only now.

Verification

• New tests. test_shared_experts_fusion_gates.py pins every family's gate branch table and which config / quantization each wrapper hands its nested body — handing over the top-level config is the failure mode, since it answers for the wrong checkpoint. test_fusion_gate_coverage.py walks the model registry and fails, naming the arch, if any entry class can reach a fusion-gated family without answering for it; the gated-class set is resolved dynamically so a wrapper around an inherited gate (DeepseekV3ForCausalLM) counts. Reverse-verified by removing a gate and confirming the arch is named. test_draft_construction_isolation.py drives the installer with stand-in model classes and covers flag seeding, draft-scope routing and restore, forward-time-read refusal, and the weight-update record.
• Unit. Models + spec suites at the stack tip: 381 passed. Each unit of the stack passes its own ratchets and config suites in isolation (482 / 482 / 523 passed).
• Full 16-partition registered CPU battery, rebased stack against a baseline built on the same base commit: zero regressions (321 failures on both sides, no aborted partitions).
• GPU e2e, byte-identical outputs against the previous shape of this change: GLM-4.7-Flash + NEXTN (default: fusion on, no disable message; --ep-size 2: two disable messages, target and draft — the loader-installed decision matches what the per-family evaluation produced) and Qwen3-Next-80B-FP8 + NEXTN (accept length 2.20). The Qwen3.5 rejection-sampling case (test_eagle_reject_sampling.py) launches with the same KV pool layout as the base commit, checked against a side-by-side run.

---

CI States

Latest PR Test (Base): 🚫 Run #31242283228
Latest PR Test (Extra): 🚫 Run #31242283198

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 21e986251a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ch-wan]

Summary

The per-runner MoE flag design (ACTIVE + speculative twin, bracketed by speculative_moe_backend_context, both-ways record_shared_experts_fusion_decision, forward-time read refusal) is sound and matches the series goal of finishing config-mutation closeout. Most model gates and layer reads migrate cleanly, and the new isolation tests cover the intended contracts. However, the prior codex finding on GLM-4 NextN is a real regression: glm4_moe_nextn.py was not migrated with glm4_moe_lite_nextn.py, so top-level weight-loader metadata can still read config intent while the decoder bakes the ACTIVE decision — a fused/unfused mismatch after target auto-disable.

Issue counts by severity

• bugs: 1
• suggestions: 2
• nits: 1

Issues outside the diff

These findings reference lines that are not present in the diff and could not be posted as inline comments:

• [bug] python/sglang/srt/models/glm4_moe_nextn.py:145 -- Glm4MoeForCausalLMNextN still sets top-level self.num_fused_shared_experts from get_exec().moe.disable_shared_experts_fusion (config bag / user intent). This PR stops writing auto-disable decisions to the bag and only records them on the ACTIVE moe flag. The decoder path (Glm4MoeDecoderLayer) is updated to is_shared_experts_fusion_disabled(), and glm4_moe_lite_nextn.py was fixed the same way, but the non-lite NextN twin was not. After a target gate auto-disables fusion (EP, quant, capability, deepep, …), bag intent can remain False while ACTIVE is True: the NextN decoder builds unfused (shared_experts module, no fused expert slot), then load_weights (via Glm4MoeForCausalLM.load_weights when self.num_fused_shared_experts > 0) remaps mlp.shared_experts.* → mlp.experts.{n_routed}.* as if fused. That is a real weight-load / structure mismatch for GLM-4.5/4.6/4.7 MTP drafts.

  • Suggestion: Mirror glm4_moe_lite_nextn.py: set self.num_fused_shared_experts from is_shared_experts_fusion_disabled() (construction-time ACTIVE), not get_exec(). Prefer also calling determine_num_fused_shared_experts before building Glm4MoeModelNextN (Deepseek NextN pattern) so a draft can refine its own decision under speculative_moe_backend_context rather than only inheriting the target's ACTIVE value.

• [suggestion] python/sglang/srt/arg_groups/overrides.py:305 -- declare_load_time_override is deleted, but draft_model_load_scope / _DRAFT_LOAD_SCOPES remain as scaffolding whose only consumer was that API. ModelRunner._draft_declaration_scope (and its docstring) still wraps every load in the now-empty scope. Dead code and stale docs increase the chance someone reintroduces bag-scoped load-time overrides against the new design.

  • Suggestion: Remove draft_model_load_scope, _DRAFT_LOAD_SCOPES, and _draft_declaration_scope (or reduce them to a no-op with an explicit "retired" comment and a follow-up), and update the skill/docstring references that still mention declare_load_time_override as a live path.

• [nit] python/sglang/srt/models/glm_ocr_nextn.py:142 -- Same bag-intent read as Issue 1 (get_exec().moe.disable_shared_experts_fusion for top-level num_fused_shared_experts). Parent GlmOcr load paths do not appear to remap shared experts from this field, so this is likely inert metadata rather than a weight-load bug. Still inconsistent with the new per-runner flag model if anything later trusts the attribute.

  • Suggestion: Switch to is_shared_experts_fusion_disabled() for consistency, or drop the field if unused.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ab1e187f7e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ch-wan]

Re-review (updated head ab1e187f7e3e)

All four prior findings on older head 21e9862 are fixed:

1. bug glm4_moe_nextn.py bag-intent / no gate before decoder — fixed (runs determine_num_fused_shared_experts() and sets num_fused_shared_experts before model build)
2. suggestion leftover draft_model_load_scope / _DRAFT_LOAD_SCOPES — fixed (gone with declare_load_time_override)
3. suggestion lite NextN no fusion gate before decoder — fixed (determine_num_fused_shared_experts(...NextN) before model)
4. nit glm_ocr_nextn.py bag-intent — fixed (is_shared_experts_fusion_disabled())

The ACTIVE + speculative leaf design (record_shared_experts_fusion_decision, speculative_moe_backend_context, forward-time refusal, GDN MTP hybrid refusal, draft update_model_fields isolation) looks correct and well covered by the isolation tests.

Issue counts by severity

• bugs: 0
• suggestions: 0
• nits: 0

Verdict: LGTM on the updated head. Prior blocking issue is resolved.