Remove prefetch for megatron_gpt_sft_model.py #7274
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1) Replace super.validation_step in inference_step with the validation_step from megatron_gpt_model but without prefetch as just one batch is passed to the validation_step in megatron_gpt_sft_model and prefetching causes an error. Signed-off-by: Abhishree <[email protected]>
aklife97
reviewed
Aug 19, 2023
aklife97
reviewed
Aug 19, 2023
| @@ -400,7 +400,20 @@ def inference_step(self, dataloader_iter, batch_idx, mode, dataloader_idx=0): | |||
| self._reconfigure_and_process_inference_batch(batch, data_cfg) | |||
| # Meta data from dataset | |||
| metadata = batch.get('metadata', [{}] * len(batch['tokens'])) | |||
| loss = super().validation_step(itertools.chain([batch]), batch_idx) | |||
There was a problem hiding this comment.
if the issue happens because of prefetch-ing when len(dataloader) ==1, what happens during GPT pre-training when val set is small enough to be just 1 batch?
|
Closing as this is addressed in: #7318 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Replaces
super().validation_stepininference_stepofmegatron_gpt_sft_model.pywith thevalidation_stepfrommegatron_gpt_modelbut without prefetch as just one batch is passed to thevalidation_stepinmegatron_gpt_sft_modeland prefetching causes an error.Collection: [Note which collection this PR will affect]
Changelog
Usage
# Add a code snippet demonstrating how to use thisBefore your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information