Skip to content

feat(training): enable MFSDP V2 gradient clipping - #5540

Draft
wujingyue wants to merge 3 commits into
NVIDIA-NeMo:mainfrom
wujingyue:agent/mfsdp-v2-clip-grad
Draft

feat(training): enable MFSDP V2 gradient clipping#5540
wujingyue wants to merge 3 commits into
NVIDIA-NeMo:mainfrom
wujingyue:agent/mfsdp-v2-clip-grad

Conversation

@wujingyue

@wujingyue wujingyue commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

  • allow clip_grad > 0 for MFSDP V2
  • let the dense and EP=2 V2 smoke tests use the normal clipping default (1.0)

Dependencies

Validation

Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
Signed-off-by: Jingyue Wu <jingyuew@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant