Skip to content

fix(model): map the fused input norm for MLA with q_lora_rank=None - #5295

Closed
bzantium wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
bzantium:fix-mla-q-proj-input-norm-mapping
Closed

fix(model): map the fused input norm for MLA with q_lora_rank=None#5295
bzantium wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
bzantium:fix-mla-q-proj-input-norm-mapping

Conversation

@bzantium

@bzantium bzantium commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Maps the fused input norm that MLA uses when q_lora_rank is None, so those checkpoints stop loading with every layer's input layernorm left at its random initialization.

Fixes #5294.

Why

With q_lora_rank=None there is no query LoRA to build, so Megatron-Core creates a single linear_q_proj and Transformer Engine fuses the input norm into it. The parameter is then

decoder.layers.*.self_attention.linear_q_proj.layer_norm_weight

and no standalone input_layernorm module exists to satisfy the registry's existing decoder.layers.*.input_layernorm.weight entry. models/deepseek/common.py already maps the weight of that single-matrix projection but not the norm that comes with it.

Nothing raises. The parameter has no mapping, so the conversion never visits it and the model trains from a partly random initialization. DeepSeek-V2 236B and V3 both set q_lora_rank, which is why the headline models never exercise this; deepseek-ai/DeepSeek-V2-Lite (27 layers) and kakaocorp/kanana-2-30b-a3b-thinking (48 layers) do not set it, and both go through this shared mapping list.

What changes

One registry entry in models/deepseek/common.py. Both spellings target the same HF tensor and only one exists for a given model, so the pre-existing input_layernorm entry is unaffected. MTP mappings are rewritten from the same dict, so MTP-enabled models pick the entry up without further changes.

The PR also guards the two loops in models/conversion/model_bridge.py that consume conversion tasks. build_conversion_tasks is declared List[None | WeightConversionTask] and does leave None slots for unmapped global parameters, but both loops read task.megatron_module directly, so an unmapped parameter surfaces as

AttributeError: 'NoneType' object has no attribute 'megatron_module'

naming no parameter, despite the builder having already warned about each one. That is what made this bug hard to place. Worth noting the more dangerous variant the same gap allows: had the mapping been wrong rather than absent, there would be no None slot and no error at all.

Tests

Three cases in TestCommonMappingSingleMatrixQProjection, no weights needed:

  • the fused linear_q_proj.layer_norm_weight resolves to input_layernorm.weight
  • the standalone input_layernorm.weight entry still resolves (regression guard for the q_lora_rank set case)
  • MTP layers inherit the new entry

Removing just the new mapping line fails the first and third and leaves the second passing, so the tests track the fix rather than the file.

tests/unit_tests/models/deepseek/test_deepseek_bridges.py passes 25/25. ruff check and ruff format clean.

Verified end to end on 8x B300, TP1/PP1/EP8, with a DeepSeek-V3-shaped 48-layer / 128-expert / MTP=3 checkpoint that sets q_lora_rank=None: 51 unmapped parameters before, 0 after, weight load completing where it previously raised.

With q_lora_rank=None there is no query LoRA, so Megatron-Core builds a single
linear_q_proj and Transformer Engine fuses the input norm into it. No standalone
input_layernorm module exists, so the registry's existing entry for it matches
nothing and every layer's input norm is left at its random initialization.

Nothing raises: the parameter simply has no mapping, so the conversion never
visits it. DeepSeek-V2 236B and V3 both set q_lora_rank, which is why the path
went unnoticed; DeepSeek-V2-Lite and kanana-2-30b-a3b-thinking do not.

Also guards the two task loops against the None slots build_conversion_tasks
already returns. Without that, an unmapped parameter surfaces as an
AttributeError naming no parameter, which hides the cause of exactly this bug.

Signed-off-by: Minho Ryu <ryumin93@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@bzantium

bzantium commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Duplicate of #5262, which I opened earlier and had lost track of. #5262 is the more complete change — it guards three task loops rather than two, including stream_weights_megatron_to_hf, and adds coverage in test_model_bridge.py as well. Closing this in favour of it. Apologies for the noise.

@bzantium bzantium closed this Aug 4, 2026
@bzantium
bzantium deleted the fix-mla-q-proj-input-norm-mapping branch August 4, 2026 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DeepSeek bridge leaves every input layernorm unmapped when q_lora_rank is None

2 participants