Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Megatron positional encoding alibi fix #5808

Merged
merged 25 commits into from
Jan 25, 2023
Merged

Megatron positional encoding alibi fix #5808

merged 25 commits into from
Jan 25, 2023

Commits on Jan 16, 2023

  1. 1. Debugging.

    @michalivne
    michalivne committed Jan 16, 2023
    Configuration menu
    Copy the full SHA
    c71992a View commit details
    Browse the repository at this point in the history

  2. 1. Debugging.

    @michalivne
    michalivne committed Jan 16, 2023
    Configuration menu
    Copy the full SHA
    9d64b29 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2023

  1. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    a26e63e View commit details
    Browse the repository at this point in the history

  2. 1. Debugging.

    @michalivne
    michalivne committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    f22428c View commit details
    Browse the repository at this point in the history

  3. 1. Debugging.

    @michalivne
    michalivne committed Jan 17, 2023
    Configuration menu
    Copy the full SHA
    3801b3d View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2023

  1. Merge branch 'r1.15.0' into megatron-positiona-encoding-alibi-fix

    @michalivne
    michalivne authored Jan 18, 2023
    Configuration menu
    Copy the full SHA
    2f013c6 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2023

  1. 1. Debugging.

    @michalivne
    michalivne committed Jan 19, 2023
    Configuration menu
    Copy the full SHA
    7dccccd View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2023

  1. 1. Fixed initialization. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Jan 23, 2023
    Configuration menu
    Copy the full SHA
    da685b9 View commit details
    Browse the repository at this point in the history

  2. 1. Debugging.

    @michalivne
    michalivne committed Jan 23, 2023
    Configuration menu
    Copy the full SHA
    9d0b420 View commit details
    Browse the repository at this point in the history

  3. 1. Debugging.

    @michalivne
    michalivne committed Jan 23, 2023
    Configuration menu
    Copy the full SHA
    2e956cb View commit details
    Browse the repository at this point in the history

  4. 1. Debugging.

    @michalivne
    michalivne committed Jan 23, 2023
    Configuration menu
    Copy the full SHA
    f3b5f23 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2023

  1. 1. Debugging.

    @michalivne
    michalivne committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    8aed4ce View commit details
    Browse the repository at this point in the history

  2. 1. Debugging.

    @michalivne
    michalivne committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    b1cb2a2 View commit details
    Browse the repository at this point in the history

  3. 1. Debugging.

    @michalivne
    michalivne committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    d6dbe35 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2023

  1. 1. Debugging.

    @michalivne
    michalivne committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    dddfd2c View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    2455d6e View commit details
    Browse the repository at this point in the history

  3. 1. Debugging.

    @michalivne
    michalivne committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    13431f8 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'megatron-positiona-encoding-alibi-fix' of github.com:NV… 

    …IDIA/NeMo into megatron-positiona-encoding-alibi-fix
    @michalivne
    michalivne committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    68013a8 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'r1.15.0' into megatron-positiona-encoding-alibi-fix

    @michalivne
    michalivne authored Jan 25, 2023
    Configuration menu
    Copy the full SHA
    109ba61 View commit details
    Browse the repository at this point in the history

  6. 1. Removed scale from ALiBi. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    0e91123 View commit details
    Browse the repository at this point in the history

  7. 1. Updated yaml and added support to control number of alibi heads. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    dafdfeb View commit details
    Browse the repository at this point in the history

  8. Merge branch 'megatron-positiona-encoding-alibi-fix' of github.com:NV… 

    …IDIA/NeMo into megatron-positiona-encoding-alibi-fix
    @michalivne
    michalivne committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    bddc931 View commit details
    Browse the repository at this point in the history

  9. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    32594c7 View commit details
    Browse the repository at this point in the history

  10. 1. Removed num_attention_heads_alibi from configs. 

    Signed-off-by: Micha Livne <[email protected]>
    @michalivne
    michalivne committed Jan 25, 2023
    Configuration menu
    Copy the full SHA
    fd7c583 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'r1.15.0' into megatron-positiona-encoding-alibi-fix

    @michalivne
    michalivne authored Jan 25, 2023
    Configuration menu
    Copy the full SHA
    9cd398f View commit details
    Browse the repository at this point in the history