Skip to content

Pass process groups to remaining GTP-backed layers - #6262

Merged
fanshiqing merged 1 commit into
NVIDIA:pull-request/6234from
yashaswikarnati:ykarnati/gtp-missing-pg-plumbing
Aug 5, 2026
Merged

Pass process groups to remaining GTP-backed layers#6262
fanshiqing merged 1 commit into
NVIDIA:pull-request/6234from
yashaswikarnati:ykarnati/gtp-missing-pg-plumbing

Conversation

@yashaswikarnati

Copy link
Copy Markdown
Contributor

What

  • Pass the owning ProcessGroupCollection to the Mamba input/output projections.
  • Pass it through the shared-expert MLP.
  • Pass it to the hybrid model embedding and output projection.

Position embeddings and other small parameters remain replicated.

Why

#6234 adds the GTP-capable layer APIs, but these MIMO-used construction sites still supplied only TP. This completes the explicit process-group path for their dense GTP-rematerialized weights without adding MPU-global reads.

Dependency

Stacked on #6234 by @fanshiqing.

Validation

  • Python compile checks for changed files
  • import sorting
  • git diff --check
  • full GPU E2E is tracked by the integration PR

Signed-off-by: ykarnati <ykarnati@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 4, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@fanshiqing
fanshiqing marked this pull request as ready for review August 5, 2026 02:33
@fanshiqing
fanshiqing requested review from a team as code owners August 5, 2026 02:33
@fanshiqing
fanshiqing merged commit 16aabfa into NVIDIA:pull-request/6234 Aug 5, 2026
2 checks passed
fanshiqing pushed a commit to fanshiqing/Megatron-LM that referenced this pull request Aug 5, 2026
Signed-off-by: ykarnati <ykarnati@nvidia.com>
fanshiqing pushed a commit to fanshiqing/Megatron-LM that referenced this pull request Aug 5, 2026
Signed-off-by: ykarnati <ykarnati@nvidia.com>
fanshiqing pushed a commit to fanshiqing/Megatron-LM that referenced this pull request Aug 5, 2026
Signed-off-by: ykarnati <ykarnati@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants