Skip to content

[Doc] Document latest Mcore toggle moe_hybridep_pad_uneven_dispatch_inputs for HybridEP THD training - #4213

Merged
yaoyu-33 merged 1 commit into
NVIDIA-NeMo:mainfrom
zhongbozhu:enable_hybridep_thd_multimodal
Jul 29, 2026
Merged

[Doc] Document latest Mcore toggle moe_hybridep_pad_uneven_dispatch_inputs for HybridEP THD training #4213
yaoyu-33 merged 1 commit into
NVIDIA-NeMo:mainfrom
zhongbozhu:enable_hybridep_thd_multimodal

Conversation

@zhongbozhu

@zhongbozhu zhongbozhu commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

What does this PR do ?

Mcore dependency PR should be merged first:

Mcore dev: NVIDIA/Megatron-LM#5048

Mcore main:

Changelog

  • Add specific line by line info of high level changes in this PR.

GitHub Actions CI

See the CI section in the Contributing doc for how to trigger the CI. A Nvidia developer will need to approve and trigger the CI for external contributors.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

If you haven't finished some of the above items you can still open "Draft" PR.

Additional Information

  • Related to # (issue)

@copy-pr-bot

copy-pr-bot Bot commented Jun 8, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@zhongbozhu zhongbozhu changed the title Enable HybridEP for THD multimodal training [DO NOT MERGE] Enable HybridEP for THD multimodal training Jun 8, 2026
@yaoyu-33 yaoyu-33 added area:training Training loop, callbacks, and runtime integration blocked Work cannot move forward until an external dependency is cleared feature New capabilities, enhancements, or enablement work needs-more-tests Requires additional L0 and L1 test coverage before merge labels Jun 8, 2026
@yaoyu-33
yaoyu-33 marked this pull request as draft June 8, 2026 20:51
Comment thread src/megatron/bridge/training/config.py Outdated
Signed-off-by: Zhongbo Zhu <zhongboz@nvidia.com>
@zhongbozhu
zhongbozhu force-pushed the enable_hybridep_thd_multimodal branch from 9e9a161 to a5ff26b Compare July 22, 2026 20:31
@zhongbozhu zhongbozhu changed the title [DO NOT MERGE] Enable HybridEP for THD multimodal training [Doc] Enable HybridEP for THD multimodal training Jul 22, 2026
@zhongbozhu zhongbozhu changed the title [Doc] Enable HybridEP for THD multimodal training [Doc] Document latest Mcore toggle moe_hybridep_pad_uneven_dispatch_inputs for HybridEP THD training Jul 22, 2026
@zhongbozhu
zhongbozhu marked this pull request as ready for review July 22, 2026 20:45
@yaoyu-33 yaoyu-33 added docs Documentation-only updates or documentation debt docs-only With great power comes great responsibility. and removed feature New capabilities, enhancements, or enablement work needs-more-tests Requires additional L0 and L1 test coverage before merge labels Jul 23, 2026
@zhongbozhu
zhongbozhu requested a review from yaoyu-33 July 28, 2026 23:12
@yaoyu-33 yaoyu-33 added the r0.6.0 Auto-cherrypick to release branch. Apply before merge; cherrypick happens after merge. label Jul 29, 2026
@yaoyu-33
yaoyu-33 merged commit 09907e7 into NVIDIA-NeMo:main Jul 29, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:training Training loop, callbacks, and runtime integration blocked Work cannot move forward until an external dependency is cleared docs Documentation-only updates or documentation debt docs-only With great power comes great responsibility. r0.6.0 Auto-cherrypick to release branch. Apply before merge; cherrypick happens after merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants