Fix gpt trainer test #6915

hsiehjackson · 2023-06-24T01:34:25Z

What does this PR do ?

Fix gpt trainer inference_step to compute generation results during validation and test
Add dataset with metadata, to output in prediction file
Add token_f1 metrics for Scrolls

Collection: [NLP]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

Signed-off-by: hsiehjackson <[email protected]>

nemo/collections/nlp/models/language_modeling/megatron_gpt_sft_model.py

examples/nlp/language_modeling/tuning/megatron_gpt_peft_eval.py

Signed-off-by: hsiehjackson <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: hsiehjackson <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: hsiehjackson <[email protected]>

…into fix-gpt-trainer-test

examples/nlp/language_modeling/tuning/megatron_gpt_peft_eval.py

nemo/collections/nlp/data/language_modeling/megatron/gpt_sft_dataset.py

nemo/collections/nlp/models/language_modeling/megatron_gpt_sft_model.py

Jenkinsfile

Signed-off-by: hsiehjackson <[email protected]>

…into fix-gpt-trainer-test

for more information, see https://pre-commit.ci

Signed-off-by: hsiehjackson <[email protected]>

nemo/collections/nlp/models/language_modeling/megatron_gpt_sft_model.py

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

…into fix-gpt-trainer-test

for more information, see https://pre-commit.ci

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

nemo/collections/nlp/data/language_modeling/megatron/gpt_sft_dataset.py

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

for more information, see https://pre-commit.ci

yidong72

Looks good overall, please don't change the default configuration behavior as the old code depends on it.

examples/nlp/language_modeling/tuning/conf/megatron_gpt_sft.yaml

yidong72 · 2023-08-09T21:06:02Z

examples/nlp/language_modeling/tuning/conf/megatron_gpt_sft.yaml

-      drop_last: True
-      context_key: 'input'
-      label_key: 'output'
+      drop_last: False


don't change the default behavior

Why we drop last if we evaluate with the validation dataset?

I have one use case that I have to drop the last because I am preparing a dataset that computes contrastive loss. I need to make sure all the batch has the same batch size.

If we set drop_last=False, we can use pad_samples_to_global_batch_size=True right? Or this doesn't fulfill your settings? https://github.com/NVIDIA/NeMo/blob/fix-gpt-trainer-test/nemo/collections/nlp/models/language_modeling/megatron_gpt_sft_model.py#L788
If we set default drop_last=True, then we may drop some samples for evaluation which may show incorrect results for comparison.

nemo/collections/nlp/models/language_modeling/megatron_gpt_sft_model.py

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

…into fix-gpt-trainer-test

* Add trainer.test() for GPT Signed-off-by: hsiehjackson <[email protected]> * Remove unused part Signed-off-by: hsiehjackson <[email protected]> * Add trainer.test() for GPT Signed-off-by: hsiehjackson <[email protected]> * Remove unused part Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix training part Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config Signed-off-by: hsiehjackson <[email protected]> * Fix references and add CI Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config error Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataset Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add metadata Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config Signed-off-by: hsiehjackson <[email protected]> * Fix empty batch Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config Signed-off-by: hsiehjackson <[email protected]> * Fix max seq length Signed-off-by: hsiehjackson <[email protected]> * Fix dataset Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataset Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add token f1 Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add FA in sft Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add inference config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bug Signed-off-by: hsiehjackson <[email protected]> * Fix pad Signed-off-by: hsiehjackson <[email protected]> * Fix num batch Signed-off-by: hsiehjackson <[email protected]> * Add query_key Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove pdb Signed-off-by: hsiehjackson <[email protected]> * Fix write json Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataset bug and refactor Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add logging for prediction Signed-off-by: hsiehjackson <[email protected]> * Fix retrain Signed-off-by: hsiehjackson <[email protected]> * Add query_key in config Signed-off-by: hsiehjackson <[email protected]> * Fix bug Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config Signed-off-by: hsiehjackson <[email protected]> * Fix bug Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add inference config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bug Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix mask Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix mask Signed-off-by: hsiehjackson <[email protected]> * Split PR Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Undo commit Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Add query_key to doc_string Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Adjust yzhang123 comments Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error and follow comments Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove query key Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Remove logic and query Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove query from model Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Remove query_key Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Fix error Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix pdb Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Add default tokens_to_generate Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Revert prompt truncate re-prompt Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * skip generation with metric loss Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bug Signed-off-by: Cheng-Ping Hsieh <[email protected]> * support GPTSFTChatDataset Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add comment Signed-off-by: Cheng-Ping Hsieh <[email protected]> --------- Signed-off-by: hsiehjackson <[email protected]> Signed-off-by: Cheng-Ping Hsieh <[email protected]> Signed-off-by: Cheng-Ping Hsieh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: dorotat <[email protected]>

* Add trainer.test() for GPT Signed-off-by: hsiehjackson <[email protected]> * Remove unused part Signed-off-by: hsiehjackson <[email protected]> * Add trainer.test() for GPT Signed-off-by: hsiehjackson <[email protected]> * Remove unused part Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix training part Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config Signed-off-by: hsiehjackson <[email protected]> * Fix references and add CI Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config error Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataset Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add metadata Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config Signed-off-by: hsiehjackson <[email protected]> * Fix empty batch Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config Signed-off-by: hsiehjackson <[email protected]> * Fix max seq length Signed-off-by: hsiehjackson <[email protected]> * Fix dataset Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataset Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add token f1 Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add FA in sft Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add inference config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bug Signed-off-by: hsiehjackson <[email protected]> * Fix pad Signed-off-by: hsiehjackson <[email protected]> * Fix num batch Signed-off-by: hsiehjackson <[email protected]> * Add query_key Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove pdb Signed-off-by: hsiehjackson <[email protected]> * Fix write json Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix dataset bug and refactor Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add logging for prediction Signed-off-by: hsiehjackson <[email protected]> * Fix retrain Signed-off-by: hsiehjackson <[email protected]> * Add query_key in config Signed-off-by: hsiehjackson <[email protected]> * Fix bug Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix config Signed-off-by: hsiehjackson <[email protected]> * Fix bug Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add inference config Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bug Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix mask Signed-off-by: hsiehjackson <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix mask Signed-off-by: hsiehjackson <[email protected]> * Split PR Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Undo commit Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Add query_key to doc_string Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Adjust yzhang123 comments Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error and follow comments Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove query key Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Remove logic and query Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove query from model Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Remove query_key Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Fix error Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix pdb Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Add default tokens_to_generate Signed-off-by: Cheng-Ping Hsieh <[email protected]> * Revert prompt truncate re-prompt Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * skip generation with metric loss Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix bug Signed-off-by: Cheng-Ping Hsieh <[email protected]> * support GPTSFTChatDataset Signed-off-by: Cheng-Ping Hsieh <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add comment Signed-off-by: Cheng-Ping Hsieh <[email protected]> --------- Signed-off-by: hsiehjackson <[email protected]> Signed-off-by: Cheng-Ping Hsieh <[email protected]> Signed-off-by: Cheng-Ping Hsieh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

hsiehjackson added 2 commits June 23, 2023 18:23

Add trainer.test() for GPT

5f0d53b

Signed-off-by: hsiehjackson <[email protected]>

Remove unused part

e6865c8

Signed-off-by: hsiehjackson <[email protected]>

hsiehjackson requested review from yzhang123 and MaximumEntropy June 24, 2023 01:34

github-actions bot added the NLP label Jun 24, 2023

hsiehjackson force-pushed the fix-gpt-trainer-test branch 2 times, most recently from 70e3747 to fadf272 Compare June 26, 2023 16:47

yzhang123 reviewed Jun 26, 2023

View reviewed changes

nemo/collections/nlp/models/language_modeling/megatron_gpt_sft_model.py Show resolved Hide resolved

yzhang123 reviewed Jun 26, 2023

View reviewed changes

examples/nlp/language_modeling/tuning/megatron_gpt_peft_eval.py Show resolved Hide resolved

hsiehjackson and others added 3 commits June 27, 2023 06:06

Add trainer.test() for GPT

de3cc19

Signed-off-by: hsiehjackson <[email protected]>

Remove unused part

87ce549

Signed-off-by: hsiehjackson <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

74c6fd8

for more information, see https://pre-commit.ci

hsiehjackson force-pushed the fix-gpt-trainer-test branch from fadf272 to 74c6fd8 Compare June 26, 2023 22:06

hsiehjackson added 2 commits June 27, 2023 12:23

Fix training part

98da36e

Signed-off-by: hsiehjackson <[email protected]>

Fix conflict

9656fa9

Signed-off-by: hsiehjackson <[email protected]>

github-actions bot added the CI label Jun 27, 2023

pre-commit-ci bot and others added 5 commits June 27, 2023 19:28

[pre-commit.ci] auto fixes from pre-commit.com hooks

7c240b3

for more information, see https://pre-commit.ci

Fix config

c22cfd6

Signed-off-by: hsiehjackson <[email protected]>

Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo …

83ff763

…into fix-gpt-trainer-test

Merge branch 'main' into fix-gpt-trainer-test

afa9030

Merge branch 'main' into fix-gpt-trainer-test

ae0f1f0

arendu self-requested a review June 27, 2023 23:36

arendu reviewed Jun 27, 2023

View reviewed changes

examples/nlp/language_modeling/tuning/megatron_gpt_peft_eval.py Outdated Show resolved Hide resolved

arendu reviewed Jun 27, 2023

View reviewed changes

nemo/collections/nlp/data/language_modeling/megatron/gpt_sft_dataset.py Outdated Show resolved Hide resolved

arendu reviewed Jun 27, 2023

View reviewed changes

nemo/collections/nlp/models/language_modeling/megatron_gpt_sft_model.py Show resolved Hide resolved

arendu reviewed Jun 28, 2023

View reviewed changes

Jenkinsfile Outdated Show resolved Hide resolved

hsiehjackson and others added 4 commits June 27, 2023 20:13

Fix references and add CI

a4588f4

Signed-off-by: hsiehjackson <[email protected]>

Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo …

689f96c

…into fix-gpt-trainer-test

[pre-commit.ci] auto fixes from pre-commit.com hooks

6d7c857

for more information, see https://pre-commit.ci

Fix config error

3bbc4e7

Signed-off-by: hsiehjackson <[email protected]>

arendu reviewed Aug 4, 2023

View reviewed changes

nemo/collections/nlp/models/language_modeling/megatron_gpt_sft_model.py Show resolved Hide resolved

hsiehjackson and others added 6 commits August 4, 2023 15:53

skip generation with metric loss

4c712ae

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo …

b99f522

…into fix-gpt-trainer-test

[pre-commit.ci] auto fixes from pre-commit.com hooks

2b59de8

for more information, see https://pre-commit.ci

Revert

ee74e54

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

Fix conflict

9b9626b

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

67ec1b2

for more information, see https://pre-commit.ci

hsiehjackson requested a review from arendu August 5, 2023 00:55

hsiehjackson and others added 4 commits August 7, 2023 10:37

Merge branch 'main' into fix-gpt-trainer-test

82a940c

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

Merge branch 'main' into fix-gpt-trainer-test

c51754e

[pre-commit.ci] auto fixes from pre-commit.com hooks

ac1241f

for more information, see https://pre-commit.ci

Fix bug

29f775c

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

ekmb previously approved these changes Aug 8, 2023

View reviewed changes

arendu previously approved these changes Aug 9, 2023

View reviewed changes

Kipok reviewed Aug 10, 2023

View reviewed changes

nemo/collections/nlp/data/language_modeling/megatron/gpt_sft_dataset.py Show resolved Hide resolved

yzhang123 previously approved these changes Aug 10, 2023

View reviewed changes

support GPTSFTChatDataset

cf2f2b9

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

hsiehjackson dismissed stale reviews from yzhang123, arendu, and ekmb via cf2f2b9 August 10, 2023 02:48

[pre-commit.ci] auto fixes from pre-commit.com hooks

6764801

for more information, see https://pre-commit.ci

yidong72 reviewed Aug 10, 2023

View reviewed changes

hsiehjackson added 2 commits August 9, 2023 20:14

Add comment

9f936c0

Signed-off-by: Cheng-Ping Hsieh <[email protected]>

Merge branch 'fix-gpt-trainer-test' of https://github.com/NVIDIA/NeMo …

3d3fccc

…into fix-gpt-trainer-test

ekmb approved these changes Aug 10, 2023

View reviewed changes

hsiehjackson merged commit e0f8b9b into main Aug 10, 2023
15 checks passed

hsiehjackson deleted the fix-gpt-trainer-test branch August 10, 2023 19:44

stevehuanghe mentioned this pull request Sep 22, 2023

placeholder from Speechllm selene to main zhehuaichen/NeMo#13

Closed

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix gpt trainer test #6915

Fix gpt trainer test #6915

hsiehjackson commented Jun 24, 2023 •

edited

Loading

yidong72 left a comment

yidong72 Aug 9, 2023

hsiehjackson Aug 10, 2023

yidong72 Aug 10, 2023

hsiehjackson Aug 10, 2023

Fix gpt trainer test #6915

Fix gpt trainer test #6915

Conversation

hsiehjackson commented Jun 24, 2023 • edited Loading

What does this PR do ?

Changelog

Usage

Before your PR is "Ready for review"

Who can review?

Additional Information

yidong72 left a comment

Choose a reason for hiding this comment

yidong72 Aug 9, 2023

Choose a reason for hiding this comment

hsiehjackson Aug 10, 2023

Choose a reason for hiding this comment

yidong72 Aug 10, 2023

Choose a reason for hiding this comment

hsiehjackson Aug 10, 2023

Choose a reason for hiding this comment

hsiehjackson commented Jun 24, 2023 •

edited

Loading