Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bounty] PyTorch & HuggingFace Interface #139

Open
wants to merge 510 commits into
base: main
Choose a base branch
from
Open

[Bounty] PyTorch & HuggingFace Interface #139

wants to merge 510 commits into from
+1,985 −11
This pull request is big! We’re only showing the most recent 250 commits.

Commits on Aug 11, 2024

  1. layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    1b27532 View commit details
    Browse the repository at this point in the history

  2. layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    0876c79 View commit details
    Browse the repository at this point in the history

  3. layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    e5c56c4 View commit details
    Browse the repository at this point in the history

  4. layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    696c3bb View commit details
    Browse the repository at this point in the history

  5. layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    e807a63 View commit details
    Browse the repository at this point in the history

  6. layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    0b589ee View commit details
    Browse the repository at this point in the history

  7. layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    ced3879 View commit details
    Browse the repository at this point in the history

  8. layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    c6693a8 View commit details
    Browse the repository at this point in the history

  9. layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    30e971d View commit details
    Browse the repository at this point in the history

  10. fixing layer issue

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    8b4e624 View commit details
    Browse the repository at this point in the history

  11. temp and layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    bcb499e View commit details
    Browse the repository at this point in the history

  12. temp and layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    724c6c4 View commit details
    Browse the repository at this point in the history

  13. temp and layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    e23f3f7 View commit details
    Browse the repository at this point in the history

  14. temp and layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    7f13a6d View commit details
    Browse the repository at this point in the history

  15. temp and layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    ec92328 View commit details
    Browse the repository at this point in the history

  16. temp and layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    fc3d224 View commit details
    Browse the repository at this point in the history

  17. temp and layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    f14a339 View commit details
    Browse the repository at this point in the history

  18. temp and layer test

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    4f4a9e1 View commit details
    Browse the repository at this point in the history

  19. change temp

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    3da44b3 View commit details
    Browse the repository at this point in the history

  20. change temp and alpha

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    0a4a003 View commit details
    Browse the repository at this point in the history

  21. change temp and alpha

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    e351501 View commit details
    Browse the repository at this point in the history

  22. change temp

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    3251567 View commit details
    Browse the repository at this point in the history

  23. change temp

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    16e4f7e View commit details
    Browse the repository at this point in the history

  24. change temp

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    6083927 View commit details
    Browse the repository at this point in the history

  25. change sampling

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    5b02fd1 View commit details
    Browse the repository at this point in the history

  26. change sampling

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    9805ac2 View commit details
    Browse the repository at this point in the history

  27. change sampling

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    8da3114 View commit details
    Browse the repository at this point in the history

  28. change sampling

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    c62dd2d View commit details
    Browse the repository at this point in the history

  29. change sampling

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    0f7f96d View commit details
    Browse the repository at this point in the history

  30. change sampling

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    fc36619 View commit details
    Browse the repository at this point in the history

  31. remove softmax

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    b5f98d5 View commit details
    Browse the repository at this point in the history

  32. remove softmax

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    52d608f View commit details
    Browse the repository at this point in the history

  33. float long issue

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    b17a9ab View commit details
    Browse the repository at this point in the history

  34. float long issue

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    69552e0 View commit details
    Browse the repository at this point in the history

  35. float long issue

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    1ee8a10 View commit details
    Browse the repository at this point in the history

  36. float long issue

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    1d9f482 View commit details
    Browse the repository at this point in the history

  37. float long issue

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    2ca9689 View commit details
    Browse the repository at this point in the history

  38. cleaning up utils.py

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    0b8c9f2 View commit details
    Browse the repository at this point in the history

  39. removing broken llama.py

    @risingsunomi
    risingsunomi committed Aug 11, 2024
    Configuration menu
    Copy the full SHA
    94de83f View commit details
    Browse the repository at this point in the history

  40. Merge pull request #2 from exo-explore/main 

    matching main
    @risingsunomi
    risingsunomi authored Aug 11, 2024
    Configuration menu
    Copy the full SHA
    f63f4b0 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2024

  1. Merge branch 'main' into main

    @risingsunomi
    risingsunomi authored Aug 22, 2024
    Configuration menu
    Copy the full SHA
    ab273d3 View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2024

  1. Merge pull request #4 from exo-explore/main 

    merge fork
    @risingsunomi
    risingsunomi authored Aug 24, 2024
    Configuration menu
    Copy the full SHA
    c365749 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #5 from exo-explore/main 

    merge with fork main
    @risingsunomi
    risingsunomi authored Aug 24, 2024
    Configuration menu
    Copy the full SHA
    0b8221f View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2024

  1. removing unittest, update inference return type, fixing converting te… 

    …nsor to numpy
    @risingsunomi
    risingsunomi committed Aug 25, 2024
    Configuration menu
    Copy the full SHA
    226a0ac View commit details
    Browse the repository at this point in the history

  2. adding nvidia quadro and t1000 support

    @risingsunomi
    risingsunomi committed Aug 25, 2024
    Configuration menu
    Copy the full SHA
    e11bebd View commit details
    Browse the repository at this point in the history

  3. updating test, updating model selection for smaller quant llama3 model

    @risingsunomi
    risingsunomi committed Aug 25, 2024
    Configuration menu
    Copy the full SHA
    778cb6e View commit details
    Browse the repository at this point in the history

  4. added updating model options to update_deps.py

    @risingsunomi
    risingsunomi committed Aug 25, 2024
    Configuration menu
    Copy the full SHA
    56aae50 View commit details
    Browse the repository at this point in the history

  5. updating inference class init to take shard, updating pytorch test_in… 

    …ference_engine.py, adding in pytorch option for inference engine
    @risingsunomi
    risingsunomi committed Aug 25, 2024
    Configuration menu
    Copy the full SHA
    7df4640 View commit details
    Browse the repository at this point in the history

  6. adding updates for inference_engine.py

    @risingsunomi
    risingsunomi committed Aug 25, 2024
    Configuration menu
    Copy the full SHA
    aa769ca View commit details
    Browse the repository at this point in the history

  7. reducing layer amount for llama3-2b-base

    @risingsunomi
    risingsunomi committed Aug 25, 2024
    Configuration menu
    Copy the full SHA
    08e8b41 View commit details
    Browse the repository at this point in the history

  8. fixing gpu tensor to numpy conversion issues, updating top_p_sampling… 

    … along with adding in torch topk, added in better random distribution selection for when top_p is too low or high, started work on forward_layer_cached but infer functions need to be changed and take any and not a string
    @risingsunomi
    risingsunomi committed Aug 25, 2024
    Configuration menu
    Copy the full SHA
    dd2812b View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. forward rewrite, adding in caching with dynamic cache, cache conversi… 

    …on from and to json
    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    7bcd35e View commit details
    Browse the repository at this point in the history

  2. updates to caching, stuck on issue with infer_prompt and infer_tensor… 

    … where data from infer_prompt is not complete
    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    3beea22 View commit details
    Browse the repository at this point in the history

  3. trying to fix infer problems

    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    87a14ca View commit details
    Browse the repository at this point in the history

  4. switched everything to use caching, did more prep for encoding the to… 

    …ken/logit coming from infer_tensor to infer_prompt, running into OOM issues trying on server
    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    356bf2f View commit details
    Browse the repository at this point in the history

  5. fixing test

    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    aa89032 View commit details
    Browse the repository at this point in the history

  6. adding init py for old python versions

    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    b9331d7 View commit details
    Browse the repository at this point in the history

  7. update readme and add in init pys

    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    2c7aa9c View commit details
    Browse the repository at this point in the history

  8. adding more tests

    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    6da3e94 View commit details
    Browse the repository at this point in the history

  9. adding more try catch to move through tests

    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    d0bc93c View commit details
    Browse the repository at this point in the history

  10. tests

    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    0e221b2 View commit details
    Browse the repository at this point in the history

  11. added position embeddings, update test

    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    9fc9fdb View commit details
    Browse the repository at this point in the history

  12. tests

    @risingsunomi
    risingsunomi committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    2635b4c View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. adding back tests

    @risingsunomi
    risingsunomi committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    86e89eb View commit details
    Browse the repository at this point in the history

  2. adding another test

    @risingsunomi
    risingsunomi committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    64fbacd View commit details
    Browse the repository at this point in the history

  3. Merge pull request #6 from exo-explore/main 

    merge fork to pr branch
    @risingsunomi
    risingsunomi authored Aug 27, 2024
    Configuration menu
    Copy the full SHA
    fb7c73f View commit details
    Browse the repository at this point in the history

  4. added gc collect to remove gpu, fixed tokenizers warning

    @risingsunomi
    risingsunomi committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    0d93130 View commit details
    Browse the repository at this point in the history

  5. fixing device

    @risingsunomi
    risingsunomi committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    0ae716d View commit details
    Browse the repository at this point in the history

  6. adding smaller model test

    @risingsunomi
    risingsunomi committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    7705639 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. testing

    @risingsunomi
    risingsunomi committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    81d597d View commit details
    Browse the repository at this point in the history

  2. added tinyllama

    @risingsunomi
    risingsunomi committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    f1d3e31 View commit details
    Browse the repository at this point in the history

  3. changing top_p

    @risingsunomi
    risingsunomi committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    bf0e606 View commit details
    Browse the repository at this point in the history

  4. updating test

    @risingsunomi
    risingsunomi committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    432efb5 View commit details
    Browse the repository at this point in the history

  5. adding A10, adding test

    @risingsunomi
    risingsunomi committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    2cdc14c View commit details
    Browse the repository at this point in the history

  6. removing reloading of shard, changing temp and top_p

    @risingsunomi
    risingsunomi committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    ed5bea7 View commit details
    Browse the repository at this point in the history

  7. Merge pull request #7 from exo-explore/main 

    merge to fork main
    @risingsunomi
    risingsunomi authored Aug 28, 2024
    Configuration menu
    Copy the full SHA
    ea41845 View commit details
    Browse the repository at this point in the history

  8. Merge pull request #8 from risingsunomi/pr139-dev 

    Pr139 dev
    @risingsunomi
    risingsunomi authored Aug 28, 2024
    Configuration menu
    Copy the full SHA
    46667b6 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2024

  1. rewrite of sharded model using new split testing of huggingface models

    @risingsunomi
    risingsunomi committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    032c9b1 View commit details
    Browse the repository at this point in the history

  2. building out new hf.py class, testing qwen and llama3 8b

    @risingsunomi
    risingsunomi committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    626b223 View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. trying to load in weights but transformers/pytorch doesnt allow that … 

    …unless wanting to rebuild the whole model
    @risingsunomi
    risingsunomi committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    f983e93 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. adding more testing, refining logit selection

    @risingsunomi
    risingsunomi committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    d142be0 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2024

  1. working split model test, updating class

    @risingsunomi
    risingsunomi committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    be8d7fb View commit details
    Browse the repository at this point in the history

  2. working on class and inference engine updates

    @risingsunomi
    risingsunomi committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    9d1ecdd View commit details
    Browse the repository at this point in the history

  3. building out inference engine test

    @risingsunomi
    risingsunomi committed Sep 15, 2024
    Configuration menu
    Copy the full SHA
    4b0df06 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. adding working tests, update to forward function to just use input_id… 

    …s, update infer to pass logits and hidden_states
    @risingsunomi
    risingsunomi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    623468c View commit details
    Browse the repository at this point in the history

  2. cleaning up code and tests, debugging and adding in cleaned up loggin… 

    …g, added model value options
    @risingsunomi
    risingsunomi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    19b322d View commit details
    Browse the repository at this point in the history

  3. getting infer and stop token issues

    @risingsunomi
    risingsunomi committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    cc2c14c View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. add tracking of next token and other logits into the full input_ids s… 

    …o it will stop, context was also dropping due to not having all the logits
    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    583629c View commit details
    Browse the repository at this point in the history

  2. grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    7ec5bb8 View commit details
    Browse the repository at this point in the history

  3. grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    5903e63 View commit details
    Browse the repository at this point in the history

  4. grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    e7a3fd0 View commit details
    Browse the repository at this point in the history

  5. grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    f6eec5a View commit details
    Browse the repository at this point in the history

  6. grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    d441a51 View commit details
    Browse the repository at this point in the history

  7. grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    e7f6dcb View commit details
    Browse the repository at this point in the history

  8. grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    ba5b005 View commit details
    Browse the repository at this point in the history

  9. grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    6242d76 View commit details
    Browse the repository at this point in the history

  10. grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    5630731 View commit details
    Browse the repository at this point in the history

  11. testing passing hidden states in inference_state

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    4a29268 View commit details
    Browse the repository at this point in the history

  12. testing passing hidden states in inference_state

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    2daf65f View commit details
    Browse the repository at this point in the history

  13. fixing scalar issue, reversing passing hidden_states

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    36d5cde View commit details
    Browse the repository at this point in the history

  14. inference bug fix, grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    6917f30 View commit details
    Browse the repository at this point in the history

  15. inference bug fix, grpc testing

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    adab336 View commit details
    Browse the repository at this point in the history

  16. fixing hf model for hidden_states

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    73146dd View commit details
    Browse the repository at this point in the history

  17. fixing hf model for hidden_states

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    929386d View commit details
    Browse the repository at this point in the history

  18. fixing hf model for hidden_states

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    32b8f67 View commit details
    Browse the repository at this point in the history

  19. fixing hf model for hidden_states

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    c86facb View commit details
    Browse the repository at this point in the history

  20. fixing hf model for hidden_states

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    d15b20d View commit details
    Browse the repository at this point in the history

  21. fixing hf model for hidden_states

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    5e41bc4 View commit details
    Browse the repository at this point in the history

  22. fixing hf model for hidden_states

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    b29c5f8 View commit details
    Browse the repository at this point in the history

  23. fixing kvcache issue

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    ddaa79c View commit details
    Browse the repository at this point in the history

  24. fixing kvcache issue

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    3164d38 View commit details
    Browse the repository at this point in the history

  25. fixing kvcache issue

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    e8532bc View commit details
    Browse the repository at this point in the history

  26. fixing kvcache issue

    @risingsunomi
    risingsunomi committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    6a5b8db View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. working on passing past input_ids between infers and nodes

    @risingsunomi
    risingsunomi committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    515687d View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. implemented infer caching and passing cache information via inference… 

    …_state
    @risingsunomi
    risingsunomi committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    92ebdd5 View commit details
    Browse the repository at this point in the history

  2. removing dynamic cache passing in inference_state as model does its o… 

    …wn, added cleaning out cached_iids when process is finished
    @risingsunomi
    risingsunomi committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    f0795bd View commit details
    Browse the repository at this point in the history

  3. removed clearning cache on infer prompt and only on finished infer te… 

    …nsor
    @risingsunomi
    risingsunomi committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    b8f15a0 View commit details
    Browse the repository at this point in the history

  4. hidden state dropping between nodes issue

    @risingsunomi
    risingsunomi committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    d0f3cb7 View commit details
    Browse the repository at this point in the history

  5. hidden state dropping between nodes issue

    @risingsunomi
    risingsunomi committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    fa6f263 View commit details
    Browse the repository at this point in the history

  6. hidden state dropping between nodes issue

    @risingsunomi
    risingsunomi committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    2b0e7b5 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' of github.com:exo-explore/exo into exo-fork-update

    @risingsunomi
    risingsunomi committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    f793c00 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'exo-fork-update' of github.com:risingsunomi/exo-nvidia … 

    …into exo-fork-update
    @risingsunomi
    risingsunomi committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    131c158 View commit details
    Browse the repository at this point in the history

  9. Merge pull request #11 from risingsunomi/exo-fork-update 

    Exo fork update
    @risingsunomi
    risingsunomi authored Sep 19, 2024
    Configuration menu
    Copy the full SHA
    43a1f61 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. Merge branch 'exo-explore:main' into main

    @risingsunomi
    risingsunomi authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    8398409 View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. Merge github.com:exo-explore/exo into exo-explore-main

    @risingsunomi
    risingsunomi committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    09572c1 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #13 from risingsunomi/exo-explore-main 

    Exo explore main fork update
    @risingsunomi
    risingsunomi authored Oct 2, 2024
    Configuration menu
    Copy the full SHA
    c861f30 View commit details
    Browse the repository at this point in the history

  3. cleaning up code, removing helpers.py

    @risingsunomi
    risingsunomi committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    cee3e31 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of github.com:risingsunomi/exo-nvidia

    @risingsunomi
    risingsunomi committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    f95942f View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2024

  1. Merge branch 'main' into HEAD

    @AlexCheema
    AlexCheema committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    c3ea732 View commit details
    Browse the repository at this point in the history

  2. adding needed libs to setup.py, fixing 4 space to 2 space issue, addi… 

    …ng in hf downloader to inference engine, testing
    @risingsunomi
    risingsunomi committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    57e14e8 View commit details
    Browse the repository at this point in the history

  3. cleaning up code, added pytorch engine to llama 3.2 1b model shard in… 

    … models.py, removed old 3.2 1b model shard, moving to test server for more vram
    @risingsunomi
    risingsunomi committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    9fe3ec6 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #14 from risingsunomi/pr139-dev-oct24 

    Pr139 dev oct24
    @risingsunomi
    risingsunomi authored Oct 6, 2024
    Configuration menu
    Copy the full SHA
    447da4a View commit details
    Browse the repository at this point in the history

  5. updating pytorch requirement

    @risingsunomi
    risingsunomi committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    b44f6e9 View commit details
    Browse the repository at this point in the history

  6. Merge pull request #15 from risingsunomi/pr139-dev-oct24 

    updating pytorch requirement
    @risingsunomi
    risingsunomi authored Oct 6, 2024
    Configuration menu
    Copy the full SHA
    6e1ab58 View commit details
    Browse the repository at this point in the history

  7. trying tokenizer fixes for llama3.1

    @risingsunomi
    risingsunomi committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    936e60a View commit details
    Browse the repository at this point in the history

  8. detecting 3.1 for adding padding token and using autotokenizer for ll… 

    …ama models
    @risingsunomi
    risingsunomi committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    43c3c62 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. updating models.py to use instruct version

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    75a29f4 View commit details
    Browse the repository at this point in the history

  2. fixing autotokenizer

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    e407404 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #16 from risingsunomi/pr139-dev-oct24 

    Pr139 dev oct24
    @risingsunomi
    risingsunomi authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    47ff4b3 View commit details
    Browse the repository at this point in the history

  4. making it so position and cache is computed every forward on hf model

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    668668f View commit details
    Browse the repository at this point in the history

  5. loading cached input_ids when passing hidden states

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    4e356f8 View commit details
    Browse the repository at this point in the history

  6. loading cached iids from infer state fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    a5ef04a View commit details
    Browse the repository at this point in the history

  7. device fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    e888baa View commit details
    Browse the repository at this point in the history

  8. position id fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    7d9eb17 View commit details
    Browse the repository at this point in the history

  9. fixing inference instance state issues between nodes

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    1198628 View commit details
    Browse the repository at this point in the history

  10. node testing

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    d25b7ac View commit details
    Browse the repository at this point in the history

  11. node inference fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    0721a4c View commit details
    Browse the repository at this point in the history

  12. node inference fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    49b682b View commit details
    Browse the repository at this point in the history

  13. node inference fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    77a52a5 View commit details
    Browse the repository at this point in the history

  14. node inference fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    2b3397f View commit details
    Browse the repository at this point in the history

  15. node inference fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    2e588af View commit details
    Browse the repository at this point in the history

  16. node inference fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    e2eba05 View commit details
    Browse the repository at this point in the history

  17. node inference fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    d7699eb View commit details
    Browse the repository at this point in the history

  18. inference between nodes fixed by always calculating position id and i… 

    …nput embed from input_ids cache, working on vram mem management
    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    bd9bf4f View commit details
    Browse the repository at this point in the history

  19. Merge pull request #17 from risingsunomi/pr139-dev-oct24 

    Pr139 dev oct24
    @risingsunomi
    risingsunomi authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    72bed37 View commit details
    Browse the repository at this point in the history

  20. cleaning up code

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    913a008 View commit details
    Browse the repository at this point in the history

  21. Merge pull request #18 from risingsunomi/pr139-dev-oct24 

    cleaning up code
    @risingsunomi
    risingsunomi authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    a1c1c76 View commit details
    Browse the repository at this point in the history

  22. Merge branch 'main' into HEAD

    @AlexCheema
    AlexCheema committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    d509e43 View commit details
    Browse the repository at this point in the history

  23. Merge branch 'main' into HEAD

    @AlexCheema
    AlexCheema committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    216e83d View commit details
    Browse the repository at this point in the history

  24. comma and other text issue fix

    @risingsunomi
    risingsunomi committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    b518f73 View commit details
    Browse the repository at this point in the history

  25. Merge pull request #19 from risingsunomi/pr139-dev-oct24 

    comma and other text issue fix
    @risingsunomi
    risingsunomi authored Oct 7, 2024
    Configuration menu
    Copy the full SHA
    296dff6 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2024

  1. adding threadpooling to forward and logit sampling

    @risingsunomi
    risingsunomi committed Oct 9, 2024
    Configuration menu
    Copy the full SHA
    9d24779 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #20 from risingsunomi/pr139-dev-oct24 

    adding threadpooling to forward and logit sampling
    @risingsunomi
    risingsunomi authored Oct 9, 2024
    Configuration menu
    Copy the full SHA
    fe6ae45 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. Merge branch 'exo-explore:main' into main

    @risingsunomi
    risingsunomi authored Oct 10, 2024
    Configuration menu
    Copy the full SHA
    2c55634 View commit details
    Browse the repository at this point in the history

  2. rename (PyTorch, pytorch) -> (Torch, torch)

    @AlexCheema
    AlexCheema committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    d4fb74f View commit details
    Browse the repository at this point in the history

  3. add ci jobs for chatgpt_api_integration_test_torch_linux_cpu and chat… 

    …gpt_api_integration_test_torch_mac
    @AlexCheema
    AlexCheema committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    edf1c3d View commit details
    Browse the repository at this point in the history

  4. add ci jobs for chatgpt_api_integration_test_torch_linux_cpu and chat… 

    …gpt_api_integration_test_torch_mac
    @AlexCheema
    AlexCheema committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    0fd6711 View commit details
    Browse the repository at this point in the history

  5. ci filters

    @AlexCheema
    AlexCheema committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    a4feeab View commit details
    Browse the repository at this point in the history

  6. rm comments

    @AlexCheema
    AlexCheema committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    55fd482 View commit details
    Browse the repository at this point in the history

  7. ci

    @AlexCheema
    AlexCheema committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    da39519 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into HEAD

    @AlexCheema
    AlexCheema committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    89f1be0 View commit details
    Browse the repository at this point in the history

  9. Merge remote-tracking branch 'origin/main' into HEAD

    @AlexCheema
    AlexCheema committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    b6f6afc View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. fixed torch device selection

    @risingsunomi
    risingsunomi committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    5eb6c34 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:risingsunomi/exo-nvidia into pr139-… 

    …dev-oct24
    @risingsunomi
    risingsunomi committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    ed64437 View commit details
    Browse the repository at this point in the history

  3. fixing imports

    @risingsunomi
    risingsunomi committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    18d41eb View commit details
    Browse the repository at this point in the history

  4. Merge pull request #21 from risingsunomi/pr139-dev-oct24 

    fixed torch device selection
    @risingsunomi
    risingsunomi authored Oct 11, 2024
    Configuration menu
    Copy the full SHA
    c73ed76 View commit details
    Browse the repository at this point in the history

  5. fixing chatgpt_api mistake

    @risingsunomi
    risingsunomi committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    9ecbf0c View commit details
    Browse the repository at this point in the history

  6. Merge branch 'exo-explore:main' into main

    @risingsunomi
    risingsunomi authored Oct 11, 2024
    Configuration menu
    Copy the full SHA
    79c9e70 View commit details
    Browse the repository at this point in the history

  7. Merge pull request #22 from risingsunomi/pr139-dev-oct24 

    fixing chatgpt_api mistake
    @risingsunomi
    risingsunomi authored Oct 11, 2024
    Configuration menu
    Copy the full SHA
    ebfd44a View commit details
    Browse the repository at this point in the history

  8. removing old pytorch folder

    @risingsunomi
    risingsunomi committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    dae2cbe View commit details
    Browse the repository at this point in the history

  9. Merge pull request #23 from risingsunomi/pr139-dev-oct24 

    removing old pytorch folder
    @risingsunomi
    risingsunomi authored Oct 11, 2024
    Configuration menu
    Copy the full SHA
    1c1dd06 View commit details
    Browse the repository at this point in the history

  10. Update README.md 

    cleaning up readme
    @risingsunomi
    risingsunomi authored Oct 11, 2024
    Configuration menu
    Copy the full SHA
    55ae027 View commit details
    Browse the repository at this point in the history

  11. set all torch models in models.py

    @AlexCheema
    AlexCheema committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    4b6a86d View commit details
    Browse the repository at this point in the history

  12. in torch, explicitly set the device when initilaizing the model

    @AlexCheema
    AlexCheema committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    830d33d View commit details
    Browse the repository at this point in the history

  13. spacing

    @AlexCheema
    AlexCheema committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    074dfe3 View commit details
    Browse the repository at this point in the history

  14. add model mlx-community/Qwen2-0.5B-Instruct-4bit

    @AlexCheema
    AlexCheema committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    d9cfcc4 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2024

  1. Merge branch 'exo-explore:main' into main

    @risingsunomi
    risingsunomi authored Oct 12, 2024
    Configuration menu
    Copy the full SHA
    c3e1934 View commit details
    Browse the repository at this point in the history

  2. code changes from PR feedback, working on splitting of weights

    @risingsunomi
    risingsunomi committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    2c056b4 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'exo-explore:main' into pr139-dev-oct24

    @risingsunomi
    risingsunomi authored Oct 12, 2024
    Configuration menu
    Copy the full SHA
    da5c28d View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2024

  1. doing more work toward individual safetensor loading, adding back dev… 

    …ice mapping auto
    @risingsunomi
    risingsunomi committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    83a723b View commit details
    Browse the repository at this point in the history

  2. working on split model, moving to server for more vram

    @risingsunomi
    risingsunomi committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    47be250 View commit details
    Browse the repository at this point in the history

  3. change to hf downloader as was not getting all safetensor files

    @risingsunomi
    risingsunomi committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    ea0d4b1 View commit details
    Browse the repository at this point in the history

  4. splitting model still work in progress as transformers still seems to… 

    … try to load more than needed even witha modified safetensor json file, finished up PR main updates but will continue on this one
    @risingsunomi
    risingsunomi committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    30b7991 View commit details
    Browse the repository at this point in the history

  5. updating readme

    @risingsunomi
    risingsunomi committed Oct 13, 2024
    Configuration menu
    Copy the full SHA
    3a2c431 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into pr139-dev-oct24

    @risingsunomi
    risingsunomi authored Oct 13, 2024
    Configuration menu
    Copy the full SHA
    4def538 View commit details
    Browse the repository at this point in the history

  7. Merge pull request #24 from risingsunomi/pr139-dev-oct24 

    Pr139 dev oct24
    @risingsunomi
    risingsunomi authored Oct 13, 2024
    Configuration menu
    Copy the full SHA
    b35224c View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. successful splitting model test with only loading needed weights, imp… 

    …lementing it in main inference code
    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    6c6e7b2 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'pr139-dev-oct24' of github.com:risingsunomi/exo-nvidia … 

    …into pr139-dev-oct24
    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    55ffdc7 View commit details
    Browse the repository at this point in the history

  3. adding model sharding to inference engine, doing testing with inferen… 

    …ce engine and sharding
    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    aacdeb5 View commit details
    Browse the repository at this point in the history

  4. fixing layer range issue

    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    ce702d1 View commit details
    Browse the repository at this point in the history

  5. fixing layer range issue

    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    e387a79 View commit details
    Browse the repository at this point in the history

  6. fixing layer range issue

    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    e0ba2bb View commit details
    Browse the repository at this point in the history

  7. checking if ram over usaage even if reducing layers on large models

    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    5b9638f View commit details
    Browse the repository at this point in the history

  8. half layer inference engine testing

    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    664f29f View commit details
    Browse the repository at this point in the history

  9. fixing layer amount with sharded modeling

    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    2591fab View commit details
    Browse the repository at this point in the history

  10. adding qwen2.5 3B for testing

    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    99dac57 View commit details
    Browse the repository at this point in the history

  11. Merge pull request #25 from risingsunomi/pr139-dev-oct24 

    Pr139 dev oct24
    @risingsunomi
    risingsunomi authored Oct 14, 2024
    Configuration menu
    Copy the full SHA
    c12526f View commit details
    Browse the repository at this point in the history

  12. updating inference engine test

    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    493cd3e View commit details
    Browse the repository at this point in the history

  13. cleaning up utils and split model

    @risingsunomi
    risingsunomi committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    de23294 View commit details
    Browse the repository at this point in the history

  14. Merge pull request #26 from risingsunomi/pr139-dev-oct24 

    Pr139 dev oct24
    @risingsunomi
    risingsunomi authored Oct 14, 2024
    Configuration menu
    Copy the full SHA
    d5a02be View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2024

  1. bugfix in llm setup

    @dtnewman
    dtnewman committed Oct 15, 2024
    Configuration menu
    Copy the full SHA
    e7470b1 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #27 from dtnewman/main 

    bugfix in llm setup
    @risingsunomi
    risingsunomi authored Oct 15, 2024
    Configuration menu
    Copy the full SHA
    fa24f46 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2024

  1. Merge remote-tracking branch 'origin/main' into HEAD

    @AlexCheema
    AlexCheema committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    5c69f3f View commit details
    Browse the repository at this point in the history

  2. handle range not satisfiable edge case

    @AlexCheema
    AlexCheema committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    f5a1cef View commit details
    Browse the repository at this point in the history

  3. updating to use automodelforcausallm instead of autoconfig

    @risingsunomi
    risingsunomi committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    751bd1c View commit details
    Browse the repository at this point in the history

  4. removing meta model

    @risingsunomi
    risingsunomi committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    7d866d8 View commit details
    Browse the repository at this point in the history

  5. updating split model test

    @risingsunomi
    risingsunomi committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    253237b View commit details
    Browse the repository at this point in the history

  6. updating split model test

    @risingsunomi
    risingsunomi committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    e46ffa4 View commit details
    Browse the repository at this point in the history

  7. automodel fix

    @risingsunomi
    risingsunomi committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    476b6ba View commit details
    Browse the repository at this point in the history

  8. fixing split model test

    @risingsunomi
    risingsunomi committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    f7e02e9 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. pytorch offload buffers error

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    bd6322f View commit details
    Browse the repository at this point in the history

  2. device_map any issue with split model

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    c51bd91 View commit details
    Browse the repository at this point in the history

  3. updating split model test

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    4a2aef4 View commit details
    Browse the repository at this point in the history

  4. fixing split model issue

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    79f0763 View commit details
    Browse the repository at this point in the history

  5. fixing node issues

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    cbbc9cf View commit details
    Browse the repository at this point in the history

  6. fixing node issues

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    58cebab View commit details
    Browse the repository at this point in the history

  7. fixing node issues

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    7f9b1bb View commit details
    Browse the repository at this point in the history

  8. fixing node issues

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    c3adec5 View commit details
    Browse the repository at this point in the history

  9. fixing node issues

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    c8e6acc View commit details
    Browse the repository at this point in the history

  10. fixing node issues, range issue

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    df028e2 View commit details
    Browse the repository at this point in the history

  11. fixing node issues, range issue

    @risingsunomi
    risingsunomi committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    e5a1939 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'main' into pr139-dev-oct24

    @risingsunomi
    risingsunomi authored Oct 17, 2024
    Configuration menu
    Copy the full SHA
    d03a85c View commit details
    Browse the repository at this point in the history

  13. Merge pull request #28 from risingsunomi/pr139-dev-oct24 

    Pr139 dev oct24
    @risingsunomi
    risingsunomi authored Oct 17, 2024
    Configuration menu
    Copy the full SHA
    69a8955 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. adding num hidden layers manipulation for all models

    @risingsunomi
    risingsunomi committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    d07b825 View commit details
    Browse the repository at this point in the history

  2. updating to use shard_num_hidden_layers

    @risingsunomi
    risingsunomi committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    a840e7f View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of github.com:risingsunomi/exo-nvidia into pr139-… 

    …dev-oct24
    @risingsunomi
    risingsunomi committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    bf5f22d View commit details
    Browse the repository at this point in the history

  4. adding in better layer manipulation

    @risingsunomi
    risingsunomi committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    52fa3f8 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2024

  1. adding in safe tensor sharding, generate model.safetensors.index.json… 

    … from single safetensor, starting safetensor sharding test
    @risingsunomi
    risingsunomi committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    ec49e31 View commit details
    Browse the repository at this point in the history

  2. implementing sharding tests, fixing bugs with safetensor recompile

    @risingsunomi
    risingsunomi committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    f45b514 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2024

  1. adding safetensor sharding, implementing it into model inference engine

    @risingsunomi
    risingsunomi committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    f90c24a View commit details
    Browse the repository at this point in the history

  2. updating backup and backup restore

    @risingsunomi
    risingsunomi committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    696c264 View commit details
    Browse the repository at this point in the history

  3. added removing backup when restoring

    @risingsunomi
    risingsunomi committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    9514e92 View commit details
    Browse the repository at this point in the history

  4. added generating weight map if none, did updates to backup and restor… 

    …e process for sharding HF safetensors
    @risingsunomi
    risingsunomi committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    d65505e View commit details
    Browse the repository at this point in the history

  5. cleaning up logging

    @risingsunomi
    risingsunomi committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    d5b6113 View commit details
    Browse the repository at this point in the history

  6. updating docstring in newest class file

    @risingsunomi
    risingsunomi committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    d2302cc View commit details
    Browse the repository at this point in the history

  7. Merge pull request #29 from risingsunomi/pr139-dev-oct24 

    Pr139 dev oct24
    @risingsunomi
    risingsunomi authored Oct 20, 2024
    Configuration menu
    Copy the full SHA
    35c32eb View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. Merge branch 'exo-explore:main' into main

    @risingsunomi
    risingsunomi authored Oct 23, 2024
    Configuration menu
    Copy the full SHA
    291aa10 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into main

    @AlexCheema
    AlexCheema authored Oct 23, 2024
    Configuration menu
    Copy the full SHA
    fcb298b View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2024

  1. merge fixing

    @risingsunomi
    risingsunomi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    6e32be6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into pr/risingsunomi/30

    @risingsunomi
    risingsunomi committed Oct 27, 2024
    Configuration menu
    Copy the full SHA
    df13fbc View commit details
    Browse the repository at this point in the history