Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Start using ModelParallelConfig from Megatron Core #6885

Merged
merged 57 commits into from
Aug 14, 2023
Merged

Start using ModelParallelConfig from Megatron Core #6885

merged 57 commits into from
Aug 14, 2023

Commits on Aug 8, 2023

  1. start adding gpt from megatron core path 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    63c127d View commit details
    Browse the repository at this point in the history

  2. set model parallel config 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    16d85c4 View commit details
    Browse the repository at this point in the history

  3. use model parallel config object 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    19e1420 View commit details
    Browse the repository at this point in the history

  4. update args 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    a309c0b View commit details
    Browse the repository at this point in the history

  5. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci @ericharper
    pre-commit-ci[bot] authored and ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    2ea9285 View commit details
    Browse the repository at this point in the history

  6. set vp size to none if it is 1 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    46ec121 View commit details
    Browse the repository at this point in the history

  7. set vp size to none if it is 1 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    575ef8a View commit details
    Browse the repository at this point in the history

  8. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci @ericharper
    pre-commit-ci[bot] authored and ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    a8b177c View commit details
    Browse the repository at this point in the history

  9. add TransformerConfig 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    a296be8 View commit details
    Browse the repository at this point in the history

  10. start updating to TransformerConfig 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    ec3c170 View commit details
    Browse the repository at this point in the history

  11. add todo 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    e2090ae View commit details
    Browse the repository at this point in the history

  12. revert to model parallel config 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    e1f38d8 View commit details
    Browse the repository at this point in the history

  13. add hidden_size to model_parallel_config 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    cbfb0d4 View commit details
    Browse the repository at this point in the history

  14. remove imports 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    cbf5036 View commit details
    Browse the repository at this point in the history

  15. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci @ericharper
    pre-commit-ci[bot] authored and ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    c6fe7ed View commit details
    Browse the repository at this point in the history

  16. remove import 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    2bd408c View commit details
    Browse the repository at this point in the history

  17. small clean up 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    06064bf View commit details
    Browse the repository at this point in the history

  18. update hidden size in peft base model, add mcore commit to jenkins 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    d8e9f4f View commit details
    Browse the repository at this point in the history

  19. update module args 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    afdf3f0 View commit details
    Browse the repository at this point in the history

  20. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci @ericharper
    pre-commit-ci[bot] authored and ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    90e8160 View commit details
    Browse the repository at this point in the history

  21. add config obj to flash attention tests 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    3f194ac View commit details
    Browse the repository at this point in the history

  22. remove args 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    95a3b68 View commit details
    Browse the repository at this point in the history

  23. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci @ericharper
    pre-commit-ci[bot] authored and ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    a0e133f View commit details
    Browse the repository at this point in the history

  24. remove sequence parallel arg 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    e2cc69a View commit details
    Browse the repository at this point in the history

  25. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci @ericharper
    pre-commit-ci[bot] authored and ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    b4b5b06 View commit details
    Browse the repository at this point in the history

  26. update args 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    bcbd787 View commit details
    Browse the repository at this point in the history

  27. add config to self 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    f31ff9f View commit details
    Browse the repository at this point in the history

  28. update args 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    dd59c1c View commit details
    Browse the repository at this point in the history

  29. update args 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    a6d124e View commit details
    Browse the repository at this point in the history

  30. update args 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    e36d8c1 View commit details
    Browse the repository at this point in the history

  31. add config to test 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    215b1e5 View commit details
    Browse the repository at this point in the history

  32. get hidden_size from config 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    61adaab View commit details
    Browse the repository at this point in the history

  33. add try except 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    8de4993 View commit details
    Browse the repository at this point in the history

  34. use default 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    48942cc View commit details
    Browse the repository at this point in the history

  35. update config with hidden size 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    7a4e884 View commit details
    Browse the repository at this point in the history

  36. remove arg 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    0c008fd View commit details
    Browse the repository at this point in the history

  37. comment out jenkins test 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    148e19a View commit details
    Browse the repository at this point in the history

  38. revert import 

    Signed-off-by: ericharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    6d6a69b View commit details
    Browse the repository at this point in the history

  39. remove optimizer_idx 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    bcc6072 View commit details
    Browse the repository at this point in the history

  40. prefetch num microbatches 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    aa5b5fb View commit details
    Browse the repository at this point in the history

  41. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    2db6c2a View commit details
    Browse the repository at this point in the history

  42. Merge branch 'main' into mcore_gpt_path

    @ericharper
    ericharper committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    e5d48ae View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2023

  1. Merge branch 'main' into mcore_gpt_path

    @ericharper
    ericharper committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    6199567 View commit details
    Browse the repository at this point in the history

  2. remove import 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    4ddc99f View commit details
    Browse the repository at this point in the history

  3. temporarily comment jenkins test 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    4551eb6 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2023

  1. pull main 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    072dcad View commit details
    Browse the repository at this point in the history

  2. update seq_length 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    a5f24e6 View commit details
    Browse the repository at this point in the history

  3. remove commented code 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    8f2e8fb View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2023

  1. update arg 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 12, 2023
    Configuration menu
    Copy the full SHA
    f07af89 View commit details
    Browse the repository at this point in the history

  2. resolve conflict 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 12, 2023
    Configuration menu
    Copy the full SHA
    c36054c View commit details
    Browse the repository at this point in the history

  3. update mbs and gbs of test 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 12, 2023
    Configuration menu
    Copy the full SHA
    7dcf6b7 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2023

  1. update batch size in test 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 13, 2023
    Configuration menu
    Copy the full SHA
    7519e0f View commit details
    Browse the repository at this point in the history

  2. fix precision in test 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 13, 2023
    Configuration menu
    Copy the full SHA
    7aa1188 View commit details
    Browse the repository at this point in the history

  3. update precision 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 13, 2023
    Configuration menu
    Copy the full SHA
    ceca1f3 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. move hidden_size out of conditional 

    Signed-off-by: eharper <[email protected]>
    @ericharper
    ericharper committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    82a55f5 View commit details
    Browse the repository at this point in the history

  2. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    0a702a0 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into mcore_gpt_path

    @ericharper
    ericharper committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    e4ba515 View commit details
    Browse the repository at this point in the history