Skip to content

Make nv-grouped-gemm truly optional - #2980

Closed
CodersAcademy006 wants to merge 7 commits into
NVIDIA:mainfrom
CodersAcademy006:fix/optional-nv-grouped-gemm-2541
Closed

Make nv-grouped-gemm truly optional#2980
CodersAcademy006 wants to merge 7 commits into
NVIDIA:mainfrom
CodersAcademy006:fix/optional-nv-grouped-gemm-2541

Conversation

@CodersAcademy006

@CodersAcademy006 CodersAcademy006 commented Jan 16, 2026

Copy link
Copy Markdown
Contributor

Fixes #2541

Summary

Makes the nv-grouped-gemm dependency truly optional by adding proper fallback handling when the package is not installed.

Changes

  • Add try/except import blocks for nv-grouped-gemm
  • Implement fallback to standard MoE operations when grouped GEMM is unavailable
  • Add clear error messages when grouped GEMM features are requested without installation
  • Update documentation to clarify optional nature of the dependency

Testing

  • Verified MoE models work correctly with and without nv-grouped-gemm installed
  • Confirmed graceful degradation when grouped GEMM features are unavailable

This PR was previously opened against the fork repository by mistake. Now correctly targeting the upstream NVIDIA/Megatron-LM repository.

@CodersAcademy006
CodersAcademy006 requested review from a team as code owners January 16, 2026 10:51
@copy-pr-bot

copy-pr-bot Bot commented Jan 16, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ko3n1g
ko3n1g requested a review from a team January 16, 2026 10:51
@CodersAcademy006 CodersAcademy006 mentioned this pull request Jan 16, 2026
Closed
@chtruong814 chtruong814 added the needs-follow-up Issue needs follow-up label Jan 18, 2026
@yaox12 yaox12 mentioned this pull request Jan 22, 2026
Closed
@yaox12 yaox12 added Expert Review [deprecated] Apply this label to indicate that your PR is ready for expert review. and removed needs-follow-up Issue needs follow-up labels Jan 22, 2026
@yaox12
yaox12 requested a review from ko3n1g January 22, 2026 03:30
@chtruong814 chtruong814 added the needs-follow-up Issue needs follow-up label Jan 22, 2026
@CodersAcademy006

CodersAcademy006 commented Mar 2, 2026

Copy link
Copy Markdown
Contributor Author

@ko3n1g — #2980 has yaox12's approval and he contributed directly to the branch. Could you take a look when you get a chance?

@CodersAcademy006

CodersAcademy006 commented Mar 5, 2026

Copy link
Copy Markdown
Contributor Author

@yaox12 also is there any other issues i can work on? Thank You.

CodersAcademy006 and others added 7 commits March 5, 2026 04:46
@CodersAcademy006
Fix NVIDIA#2541: Make nv-grouped-gemm truly optional
f583c87 
- Move nv-grouped-gemm from dev/lts extras to new moe extra
- Users can now install megatron-core[dev] without build failures
- Add comprehensive error messages with installation instructions
- Update README with MoE dependencies documentation

This resolves the installation failure when nv-grouped-gemm cannot build
due to missing CUTLASS headers. Users who need MoE with grouped GEMM can
now explicitly install it with megatron-core[moe], while others can
install dev/lts extras without encountering build errors.

Fixes NVIDIA#2541
@yaox12 @CodersAcademy006
Update grouped_gemm_util.py
5f41dcb
@yaox12 @CodersAcademy006
Update README.md
e8a10dc
@yaox12 @CodersAcademy006
Update grouped_gemm_util.py
09a7c52
@yaox12 @CodersAcademy006
Update grouped_gemm_util.py
8a8ecae
@yaox12 @CodersAcademy006
Update grouped_gemm_util.py
99b0aa3
@yaox12 @CodersAcademy006
Update README.md
4207a86
@CodersAcademy006
CodersAcademy006 force-pushed the fix/optional-nv-grouped-gemm-2541 branch from c92bccc to 4207a86 Compare March 5, 2026 04:46
@yaox12

yaox12 commented Mar 5, 2026

Copy link
Copy Markdown
Member

Plan to fully remove nv-grouped-gemm #3711.

@yaox12

yaox12 commented Mar 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Close this PR as #3770 is merged.

@yaox12 yaox12 closed this Mar 20, 2026
@chtruong814 chtruong814 removed the needs-follow-up Issue needs follow-up label Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yaox12 yaox12 yaox12 approved these changes
@ko3n1g ko3n1g Awaiting requested review from ko3n1g

Assignees

@yaox12 yaox12

Labels

community-request Expert Review [deprecated] Apply this label to indicate that your PR is ready for expert review.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Cannot install megatron-core[dev] - nv-grouped-gemm build error

4 participants

@CodersAcademy006 @yaox12 @ko3n1g @chtruong814