[mcore] add offload param and opt function for magetron#1162
Merged
ETOgaosion merged 26 commits intoverl-project:mainfrom Apr 26, 2025
Merged
[mcore] add offload param and opt function for magetron#1162ETOgaosion merged 26 commits intoverl-project:mainfrom
ETOgaosion merged 26 commits intoverl-project:mainfrom
Conversation
BearBiscuit05
changed the title
BearBiscuit05
changed the title
Collaborator
|
thanks for great work and will finish the testing on serval larger models like Qwen72b, Mixtral etc by the end of this week. cc: @ann-qin-lu |
Collaborator
Author
thx, but please note that there is currently an issue with the logger printing for GPU memory, mainly because vllm's memory release cannot be detected by torch. If you want to fix the issue, you can check PR #1118 |
ETOgaosion
approved these changes
Apr 25, 2025
ccclyu
approved these changes
Apr 26, 2025
Collaborator
ccclyu
left a comment
ccclyu
left a comment
There was a problem hiding this comment.
LGTM. can first merge for feature dev.
Collaborator
Thanks for approving. Any suggestions for future refactorization are appreciated. |
Collaborator
|
@BearBiscuit05 Remember to add some documentation if changing configurations please~ |
ScottCTD
pushed a commit
to ScottCTD/verl
that referenced
this pull request
May 5, 2025
…#1162) ## Motivation This is a PR that supports offload in Megatron. Currently, parameters, gradients, and optimizers can be offloaded to the CPU when not needed. I have successfully tested the feasibility of the function using the memory snap tool. Further accuracy testing is still in progress. ## TODO - [x] Accuracy testing
chenjiaoAngel
added a commit
to chenjiaoAngel/verl
that referenced
this pull request
Nov 14, 2025
…#1162) ## Motivation This is a PR that supports offload in Megatron. Currently, parameters, gradients, and optimizers can be offloaded to the CPU when not needed. I have successfully tested the feasibility of the function using the memory snap tool. Further accuracy testing is still in progress. ## TODO - [x] Accuracy testing
TimurTaepov
pushed a commit
to giorgossideris/verl
that referenced
this pull request
Dec 20, 2025
…#1162) ## Motivation This is a PR that supports offload in Megatron. Currently, parameters, gradients, and optimizers can be offloaded to the CPU when not needed. I have successfully tested the feasibility of the function using the memory snap tool. Further accuracy testing is still in progress. ## TODO - [x] Accuracy testing
vyomakesh0728
added a commit
to vyomakesh0728/verl
that referenced
this pull request
Jan 22, 2026
…#1162) ## Motivation This is a PR that supports offload in Megatron. Currently, parameters, gradients, and optimizers can be offloaded to the CPU when not needed. I have successfully tested the feasibility of the function using the memory snap tool. Further accuracy testing is still in progress. ## TODO - [x] Accuracy testing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
This is a PR that supports offload in Megatron. Currently, parameters, gradients, and optimizers can be offloaded to the CPU when not needed. I have successfully tested the feasibility of the function using the memory snap tool. Further accuracy testing is still in progress.
TODO