Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RoPE length extrapolation with interpolation #7005

Merged
merged 16 commits into from
Jul 11, 2023
Merged

RoPE length extrapolation with interpolation #7005

merged 16 commits into from
Jul 11, 2023

Commits on Jun 28, 2023

  1. Push changes 

    Signed-off-by: MaximumEntropy <[email protected]>
    @MaximumEntropy
    MaximumEntropy committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    b9287e3 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2023

  1. Fixes 

    Signed-off-by: MaximumEntropy <[email protected]>
    @MaximumEntropy
    MaximumEntropy committed Jun 29, 2023
    Configuration menu
    Copy the full SHA
    0a4574b View commit details
    Browse the repository at this point in the history

  2. add continue training script 

    Signed-off-by: MaximumEntropy <[email protected]>
    @MaximumEntropy
    MaximumEntropy committed Jun 29, 2023
    Configuration menu
    Copy the full SHA
    6e79e92 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2023

  1. [WIP] nonlinear interp 

    Signed-off-by: MaximumEntropy <[email protected]>
    @MaximumEntropy
    MaximumEntropy committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    363831e View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2023

  1. Fix 

    Signed-off-by: MaximumEntropy <[email protected]>
    @MaximumEntropy
    MaximumEntropy committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    95282d4 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2023

  1. override encoder_seq_len 

    Signed-off-by: MaximumEntropy <[email protected]>
    @MaximumEntropy
    MaximumEntropy committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    21739f0 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2023

  1. Remove nonlinear 

    Signed-off-by: MaximumEntropy <[email protected]>
    @MaximumEntropy
    MaximumEntropy committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    9cf880a View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    f170ed2 View commit details
    Browse the repository at this point in the history

  3. sft with pi (#7006) 

    * sft with pi

Signed-off-by: Evelina <[email protected]>

* update values only if not None"

Signed-off-by: Evelina <[email protected]>

---------

Signed-off-by: Evelina <[email protected]>
    @ekmb
    ekmb committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    5daffb7 View commit details
    Browse the repository at this point in the history

  4. Address comments 

    Signed-off-by: MaximumEntropy <[email protected]>
    @MaximumEntropy
    MaximumEntropy committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    9a6fab2 View commit details
    Browse the repository at this point in the history

  5. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    6d5f75e View commit details
    Browse the repository at this point in the history

  6. Add info 

    Signed-off-by: MaximumEntropy <[email protected]>
    @MaximumEntropy
    MaximumEntropy committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    f1d7cbe View commit details
    Browse the repository at this point in the history

  7. Merge branch 'sandeepsub/rope_interpolate' of github.com:NVIDIA/NeMo … 

    …into sandeepsub/rope_interpolate
    @MaximumEntropy
    MaximumEntropy committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    61ac17f View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into sandeepsub/rope_interpolate

    @MaximumEntropy
    MaximumEntropy committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    bc6fe06 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2023

  1. Empty 

    Signed-off-by: MaximumEntropy <[email protected]>
    @MaximumEntropy
    MaximumEntropy committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    895d8bb View commit details
    Browse the repository at this point in the history

  2. Merge branch 'sandeepsub/rope_interpolate' of github.com:NVIDIA/NeMo … 

    …into sandeepsub/rope_interpolate
    @MaximumEntropy
    MaximumEntropy committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    b60be05 View commit details
    Browse the repository at this point in the history