Skip to content

fix(modelopt): AUT-854 respect moe_router_dtype in ptq HF import - #5830

Closed
svcnemo-autobot wants to merge 2 commits into
NVIDIA:mainfrom
svcnemo-autobot:fix/ptq-moe-router-dtype
Closed

fix(modelopt): AUT-854 respect moe_router_dtype in ptq HF import#5830
svcnemo-autobot wants to merge 2 commits into
NVIDIA:mainfrom
svcnemo-autobot:fix/ptq-moe-router-dtype

Conversation

@svcnemo-autobot

Copy link
Copy Markdown
Collaborator

Background

Branched off PR #5810. The ModelOpt PTQ example downcasts the MoE router because the HF→Mcore conversion does not receive moe_router_dtype from the model args.

What changed

Pass moe_router_dtype through to import_mcore_gpt_from_hf when the importer supports it and the arg is set.

Details

examples/post_training/modelopt/quantize.py — add a guarded import_kwargs.update({"moe_router_dtype": args.moe_router_dtype}), mirroring the existing trust_remote_code signature-guard so older importers stay compatible.

Tested

Content is byte-identical to PR #5810's head (2daf9d0), whose DCO + check-approval are green. File is outside the megatron/core+tests/ autoformat scope, so linting is unaffected. Watching CI on this fork PR.

@copy-pr-bot

copy-pr-bot Bot commented Jul 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@svcnvidia-nemo-ci
svcnvidia-nemo-ci marked this pull request as draft July 15, 2026 16:05
@github-actions

Copy link
Copy Markdown
Contributor

This PR has been automatically converted to draft because all PRs must start as drafts.

When you are ready for review, click Ready for Review to begin the review process. This will:

  1. Add the oncall reviewer (optional reviewer)
  2. Add required review teams based on your changes

See the contribution guide for more details.

@svcnemo-autobot

Copy link
Copy Markdown
Collaborator Author

/ok to test 2daf9d0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants