Skip to content

fix(mistral3): preserve medium VLM checkpoint layout - #2758

Merged
akoumpa merged 1 commit into
mainfrom
akoumpa/fix/mistral3-vlm-peft-load
Jun 24, 2026
Merged

fix(mistral3): preserve medium VLM checkpoint layout#2758
akoumpa merged 1 commit into
mainfrom
akoumpa/fix/mistral3-vlm-peft-load

Conversation

@akoumpa

@akoumpa akoumpa commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Summary

  • preserve identity full-VLM checkpoint key layout for the Mistral-Medium-3.5 128B FP8 config
  • keep the nested language_model.model.* mapping for the smaller Ministral/Devstral-style layouts
  • drive the layout choice from config.text_config.model_type plus config.text_config.num_hidden_layers, not from the repo id

Failure

The regression showed up while loading the base model for mistral3p5_128b_medpix_lora:

RuntimeError: Missing key in checkpoint state_dict: language_model.model.embed_tokens.weight.

Mistral-Medium-3.5-128B stores full-VLM body keys in the HF/runtime layout (model.language_model.*, model.vision_tower.*, model.multi_modal_projector.*), so remapping those destination keys to language_model.model.* made DCP request keys that are absent from that checkpoint.

Validation

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>
@akoumpa
akoumpa requested a review from a team as a code owner June 24, 2026 18:28
@copy-pr-bot

copy-pr-bot Bot commented Jun 24, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@akoumpa

akoumpa commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Additional nemo-ci trigger for the original SFT MedPix job requested from https://gitlab-master.nvidia.com/dl/JoC/nemo-ci/-/jobs/346789236.

Current state: root pipeline is running; automodel_test is still waiting on the root build stage, so the target leaf job has not materialized yet.

@akoumpa akoumpa added the r0.5.0 Auto-cherrypick to release branch. Apply before merge; cherrypick happens after merge. label Jun 24, 2026
@akoumpa

akoumpa commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

Superseding the earlier retrigger for mistral3p5_128b_medpix:

The leaf job is currently running.

@akoumpa

akoumpa commented Jun 24, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test ffeff89

@athitten athitten left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you @akoumpa !

@akoumpa
akoumpa enabled auto-merge (squash) June 24, 2026 20:47
@akoumpa
akoumpa merged commit a482b9c into main Jun 24, 2026
80 checks passed
@akoumpa
akoumpa deleted the akoumpa/fix/mistral3-vlm-peft-load branch June 24, 2026 22:20
akoumpa added a commit that referenced this pull request Jun 25, 2026
…o `r0.5.0` (#2762)

fix(mistral3): preserve medium VLM checkpoint layout (#2758)

Signed-off-by: Alexandros Koumparoulis <akoumparouli@nvidia.com>
Signed-off-by: NeMo Bot <nemo-bot@nvidia.com>
Co-authored-by: Alexandros Koumparoulis <153118171+akoumpa@users.noreply.github.com>
@akoumpa

akoumpa commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

Final follow-up for the extra mistral3p5_128b_medpix retest: passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

r0.5.0 Auto-cherrypick to release branch. Apply before merge; cherrypick happens after merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants