Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Megatron BART #3666

Merged
merged 82 commits into from
Mar 30, 2022
Merged

Megatron BART #3666

merged 82 commits into from
Mar 30, 2022

Commits on Feb 14, 2022

  1. 1. Added skeleton. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Feb 14, 2022
    Configuration menu
    Copy the full SHA
    6334b94 View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2022

  1. Merge branch 'main' of github.com:michalivne/NeMo into megatron-bart

    @michalivne
    michalivne committed Feb 28, 2022
    Configuration menu
    Copy the full SHA
    1afa361 View commit details
    Browse the repository at this point in the history

  2. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Feb 28, 2022
    Configuration menu
    Copy the full SHA
    8fbfab3 View commit details
    Browse the repository at this point in the history

  3. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Feb 28, 2022
    Configuration menu
    Copy the full SHA
    60497af View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2022

  1. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 1, 2022
    Configuration menu
    Copy the full SHA
    1411fa9 View commit details
    Browse the repository at this point in the history

  2. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 1, 2022
    Configuration menu
    Copy the full SHA
    b0017f5 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'megatron-bart' of github.com:michalivne/NeMo into megat… 

    …ron-bart
    @michalivne
    michalivne committed Mar 1, 2022
    Configuration menu
    Copy the full SHA
    c84dbb4 View commit details
    Browse the repository at this point in the history

Commits on Mar 7, 2022

  1. Merge branch 'main' of https://github.com/NVIDIA/NeMo into megatron-bart

    @michalivne
    michalivne committed Mar 7, 2022
    Configuration menu
    Copy the full SHA
    8eab12d View commit details
    Browse the repository at this point in the history

  2. 1. Added delete_mask_prob to allow mask deletion. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 7, 2022
    Configuration menu
    Copy the full SHA
    9c7dac6 View commit details
    Browse the repository at this point in the history

  3. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 7, 2022
    Configuration menu
    Copy the full SHA
    5825669 View commit details
    Browse the repository at this point in the history

  4. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 7, 2022
    Configuration menu
    Copy the full SHA
    b8cbfad View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2022

  1. 1. Fixed style. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 8, 2022
    Configuration menu
    Copy the full SHA
    daf9891 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 8, 2022
    Configuration menu
    Copy the full SHA
    db88329 View commit details
    Browse the repository at this point in the history

  3. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 8, 2022
    Configuration menu
    Copy the full SHA
    c255086 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'megatron-bart' of github.com:michalivne/NeMo into megat… 

    …ron-bart
    @michalivne
    michalivne committed Mar 8, 2022
    Configuration menu
    Copy the full SHA
    d6eb5d5 View commit details
    Browse the repository at this point in the history

  5. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 8, 2022
    Configuration menu
    Copy the full SHA
    b2efd2e View commit details
    Browse the repository at this point in the history

  6. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 8, 2022
    Configuration menu
    Copy the full SHA
    e370f3a View commit details
    Browse the repository at this point in the history

  7. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 8, 2022
    Configuration menu
    Copy the full SHA
    f7223b1 View commit details
    Browse the repository at this point in the history

  8. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 8, 2022
    Configuration menu
    Copy the full SHA
    31e536f View commit details
    Browse the repository at this point in the history

  9. 1. Fixed BART data augmentation 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 8, 2022
    Configuration menu
    Copy the full SHA
    964a86a View commit details
    Browse the repository at this point in the history

  10. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 8, 2022
    Configuration menu
    Copy the full SHA
    8d4c58b View commit details
    Browse the repository at this point in the history

Commits on Mar 9, 2022

  1. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 9, 2022
    Configuration menu
    Copy the full SHA
    ae24a88 View commit details
    Browse the repository at this point in the history

  2. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 9, 2022
    Configuration menu
    Copy the full SHA
    8112cd6 View commit details
    Browse the repository at this point in the history

  3. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 9, 2022
    Configuration menu
    Copy the full SHA
    1d726b3 View commit details
    Browse the repository at this point in the history

  4. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 9, 2022
    Configuration menu
    Copy the full SHA
    5b2a0fb View commit details
    Browse the repository at this point in the history

  5. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 9, 2022
    Configuration menu
    Copy the full SHA
    51c4dd9 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 9, 2022
    Configuration menu
    Copy the full SHA
    0c127f6 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 9, 2022
    Configuration menu
    Copy the full SHA
    d3ad354 View commit details
    Browse the repository at this point in the history

Commits on Mar 10, 2022

  1. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 10, 2022
    Configuration menu
    Copy the full SHA
    8ce30e6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'megatron-bart' of github.com:michalivne/NeMo into megat… 

    …ron-bart
    @michalivne
    michalivne committed Mar 10, 2022
    Configuration menu
    Copy the full SHA
    f71bf25 View commit details
    Browse the repository at this point in the history

  3. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 10, 2022
    Configuration menu
    Copy the full SHA
    ef3e4b7 View commit details
    Browse the repository at this point in the history

  4. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 10, 2022
    Configuration menu
    Copy the full SHA
    ca079ab View commit details
    Browse the repository at this point in the history

  5. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 10, 2022
    Configuration menu
    Copy the full SHA
    ca30763 View commit details
    Browse the repository at this point in the history

  6. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 10, 2022
    Configuration menu
    Copy the full SHA
    35c4612 View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2022

  1. 1. Added token deletion, and text infilling. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 11, 2022
    Configuration menu
    Copy the full SHA
    ac33d99 View commit details
    Browse the repository at this point in the history

  2. 1. Added delete_mask_prob config. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 11, 2022
    Configuration menu
    Copy the full SHA
    be584c4 View commit details
    Browse the repository at this point in the history

  3. 1. Fixed style. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 11, 2022
    Configuration menu
    Copy the full SHA
    541eea5 View commit details
    Browse the repository at this point in the history

  4. 1. Added Jenkins test. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 11, 2022
    Configuration menu
    Copy the full SHA
    5fe6c25 View commit details
    Browse the repository at this point in the history

  5. 1. Updated docs. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 11, 2022
    Configuration menu
    Copy the full SHA
    87b22a6 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'megatron-bart' of github.com:michalivne/NeMo into megat… 

    …ron-bart
    @michalivne
    michalivne committed Mar 11, 2022
    Configuration menu
    Copy the full SHA
    a9849cb View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 11, 2022
    Configuration menu
    Copy the full SHA
    7492018 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2022

  1. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 14, 2022
    Configuration menu
    Copy the full SHA
    61badf7 View commit details
    Browse the repository at this point in the history

Commits on Mar 16, 2022

  1. Merge branch 'main' into megatron-bart

    @ericharper
    ericharper authored Mar 16, 2022
    Configuration menu
    Copy the full SHA
    babcda0 View commit details
    Browse the repository at this point in the history

Commits on Mar 17, 2022

  1. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 17, 2022
    Configuration menu
    Copy the full SHA
    7146917 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 17, 2022
    Configuration menu
    Copy the full SHA
    28d9c05 View commit details
    Browse the repository at this point in the history

  3. 1. Fixed BART data preparation when masked_lm_prob = 0 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 17, 2022
    Configuration menu
    Copy the full SHA
    a5c089a View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2022

  1. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 18, 2022
    Configuration menu
    Copy the full SHA
    cd277b9 View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2022

  1. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 20, 2022
    Configuration menu
    Copy the full SHA
    7a9396f View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2022

  1. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 23, 2022
    Configuration menu
    Copy the full SHA
    3d0bc92 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 23, 2022
    Configuration menu
    Copy the full SHA
    f6878f7 View commit details
    Browse the repository at this point in the history

Commits on Mar 24, 2022

  1. 1. BARTDataset now inherits from T5Dataset with heavy code reuse. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    be4d9d9 View commit details
    Browse the repository at this point in the history

  2. 1. MegatronBARTModel inhertis from MegatronT5Model with heavy code re… 

    …use.

Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    abdc992 View commit details
    Browse the repository at this point in the history

  3. 1. Fixed style. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 24, 2022
    Configuration menu
    Copy the full SHA
    95ded9d View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2022

  1. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 25, 2022
    Configuration menu
    Copy the full SHA
    9a63350 View commit details
    Browse the repository at this point in the history

  2. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 25, 2022
    Configuration menu
    Copy the full SHA
    e48fe1c View commit details
    Browse the repository at this point in the history

  3. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 25, 2022
    Configuration menu
    Copy the full SHA
    3e3a3d2 View commit details
    Browse the repository at this point in the history

  4. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 25, 2022
    Configuration menu
    Copy the full SHA
    a08df27 View commit details
    Browse the repository at this point in the history

  5. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 25, 2022
    Configuration menu
    Copy the full SHA
    278c306 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into megatron-bart

    @MaximumEntropy
    MaximumEntropy authored Mar 25, 2022
    Configuration menu
    Copy the full SHA
    de24e95 View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2022

  1. Merge branch 'main' into megatron-bart

    @ericharper
    ericharper authored Mar 27, 2022
    Configuration menu
    Copy the full SHA
    898354c View commit details
    Browse the repository at this point in the history

  2. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 27, 2022
    Configuration menu
    Copy the full SHA
    1ebc425 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2022

  1. Merge branch 'main' into megatron-bart

    @ericharper
    ericharper authored Mar 28, 2022
    Configuration menu
    Copy the full SHA
    2cca343 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2022

  1. Merge branch 'megatron-bart' of github.com:michalivne/NeMo into megat… 

    …ron-bart
    @michalivne
    michalivne committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    33bd185 View commit details
    Browse the repository at this point in the history

  2. 1. Fixed inheritance bugs (BART from T5) 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    99e5b08 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 29, 2022
    Configuration menu
    Copy the full SHA
    7d574ef View commit details
    Browse the repository at this point in the history

  4. 1. Fixed style 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    98e5d32 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into megatron-bart

    @MaximumEntropy
    MaximumEntropy authored Mar 29, 2022
    Configuration menu
    Copy the full SHA
    94ff752 View commit details
    Browse the repository at this point in the history

  6. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    caf2e17 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'megatron-bart' of github.com:michalivne/NeMo into megat… 

    …ron-bart
    @michalivne
    michalivne committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    9200de3 View commit details
    Browse the repository at this point in the history

  8. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    7a642d1 View commit details
    Browse the repository at this point in the history

  9. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    9a26836 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 29, 2022
    Configuration menu
    Copy the full SHA
    58024a0 View commit details
    Browse the repository at this point in the history

  11. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 29, 2022
    Configuration menu
    Copy the full SHA
    8419c3c View commit details
    Browse the repository at this point in the history

  12. Merge branch 'main' into megatron-bart

    @michalivne
    michalivne authored Mar 29, 2022
    Configuration menu
    Copy the full SHA
    6abf603 View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2022

  1. Merge branch 'main' into megatron-bart

    @ericharper
    ericharper authored Mar 30, 2022
    Configuration menu
    Copy the full SHA
    0b08d2a View commit details
    Browse the repository at this point in the history

  2. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 30, 2022
    Configuration menu
    Copy the full SHA
    40ad596 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'megatron-bart' of github.com:michalivne/NeMo into megat… 

    …ron-bart
    @michalivne
    michalivne committed Mar 30, 2022
    Configuration menu
    Copy the full SHA
    6c6b7fd View commit details
    Browse the repository at this point in the history

  4. 1. Fixed style. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 30, 2022
    Configuration menu
    Copy the full SHA
    c23b980 View commit details
    Browse the repository at this point in the history

  5. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 30, 2022
    Configuration menu
    Copy the full SHA
    d136dba View commit details
    Browse the repository at this point in the history

  6. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 30, 2022
    Configuration menu
    Copy the full SHA
    69f020c View commit details
    Browse the repository at this point in the history

  7. 1. Debugging. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Mar 30, 2022
    Configuration menu
    Copy the full SHA
    1c94524 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into megatron-bart

    @ericharper
    ericharper authored Mar 30, 2022
    Configuration menu
    Copy the full SHA
    81e5065 View commit details
    Browse the repository at this point in the history