Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dataloader #83

Merged
merged 58 commits into from
Jul 24, 2024
Merged

Dataloader #83

merged 58 commits into from
Jul 24, 2024

Commits on Jun 25, 2024

  1. Initial implementation with hydra

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    dfda24f View commit details
    Browse the repository at this point in the history

  2. Test CI for config yaml

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    50c63c2 View commit details
    Browse the repository at this point in the history

  3. Testing CI for hydra

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    c93c335 View commit details
    Browse the repository at this point in the history

  4. Try pip requirements.txt

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    13c5204 View commit details
    Browse the repository at this point in the history

  5. Try pip instead of conda for docker

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    7ebded9 View commit details
    Browse the repository at this point in the history

  6. Docker build on GitHub

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    85ff713 View commit details
    Browse the repository at this point in the history

  7. Copy requirements.txt file before installing on Docker container

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    e607f98 View commit details
    Browse the repository at this point in the history

  8. Copy requirements.txt

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    9913d92 View commit details
    Browse the repository at this point in the history

  9. Copy requirements.txt

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    111b428 View commit details
    Browse the repository at this point in the history

  10. Trying with user flag

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    3606790 View commit details
    Browse the repository at this point in the history

  11. Trying Python 3.11

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    b6cea3e View commit details
    Browse the repository at this point in the history

  12. Test CircleCI with ghcr container image

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    f9cb06d View commit details
    Browse the repository at this point in the history

  13. GitHub Actions workflow to test training GNS

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    5d8ce23 View commit details
    Browse the repository at this point in the history

  14. Updated dockerfile with paths and env

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    b230de8 View commit details
    Browse the repository at this point in the history

  15. Modify workflow to run training

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    20fa700 View commit details
    Browse the repository at this point in the history

  16. Add at least one epoch to run when nsteps is fewer than 1 epoch steps

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    7764f9d View commit details
    Browse the repository at this point in the history

  17. Train GNS action

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    1e612df View commit details
    Browse the repository at this point in the history

  18. Test without docker pull on CircleCI

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    1f0b8cd View commit details
    Browse the repository at this point in the history

  19. Specify path and branches

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    8c0ddad View commit details
    Browse the repository at this point in the history

  20. Fix path to GNS sample output

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    4a7e5c4 View commit details
    Browse the repository at this point in the history

  21. Worflow runs on Github and remove conda on circleci

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    c5bcc44 View commit details
    Browse the repository at this point in the history

  22. No black check

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    dc27691 View commit details
    Browse the repository at this point in the history

  23. Only try to build container if specific files have changed

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    5329b78 View commit details
    Browse the repository at this point in the history

  24. Fix resume training and README

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    a97cd5e View commit details
    Browse the repository at this point in the history

  25. Reduce number of steps to 100 for testing

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    170037b View commit details
    Browse the repository at this point in the history

  26. Refactor constants to data

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    40b4919 View commit details
    Browse the repository at this point in the history

  27. Remove on PR

    @kks32
    kks32 committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    c6b092d View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Add config to tensorboard writer

    @kks32
    kks32 committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    14f3b9e View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Particle data loader

    @kks32
    kks32 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    5bab43b View commit details
    Browse the repository at this point in the history

  2. Add tests for data loader

    @kks32
    kks32 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    8a3a540 View commit details
    Browse the repository at this point in the history

  3. Use the correct container

    @kks32
    kks32 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    29574a7 View commit details
    Browse the repository at this point in the history

  4. Remove CircleCI

    @kks32
    kks32 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    402e677 View commit details
    Browse the repository at this point in the history

  5. Reduce nsteps to 10 for CI

    @kks32
    kks32 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    e28275c View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. Resolve merge conflict with config branch

    @kks32
    kks32 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e3283cf View commit details
    Browse the repository at this point in the history

  2. multi node parallel

    Sikan Li committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    3d99936 View commit details
    Browse the repository at this point in the history

  3. resolve conflict

    Sikan Li committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    6d6b1ca View commit details
    Browse the repository at this point in the history

  4. resolve confligt

    Sikan Li committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    8f05d50 View commit details
    Browse the repository at this point in the history

  5. change GPU container

    Sikan Li committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    66289a5 View commit details
    Browse the repository at this point in the history

  6. update scripts and readme

    Sikan Li committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    9a8ea12 View commit details
    Browse the repository at this point in the history

  7. n_gpus

    Sikan Li committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    79cd9f4 View commit details
    Browse the repository at this point in the history

  8. update Dockerfile

    Sikan Li committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    748b9f4 View commit details
    Browse the repository at this point in the history

  9. reformat

    Sikan Li committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    a74ced0 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2024

  1. Remove unused dataloader

    @kks32
    kks32 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    768702a View commit details
    Browse the repository at this point in the history

  2. GPU cocntainer

    @kks32
    kks32 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    0a28073 View commit details
    Browse the repository at this point in the history

  3. Remove blank lines in GPU container

    @kks32
    kks32 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    b1081b9 View commit details
    Browse the repository at this point in the history

  4. Update README with container image

    @kks32
    kks32 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    3202caf View commit details
    Browse the repository at this point in the history

  5. Add GitHub badge

    @kks32
    kks32 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    291ce9a View commit details
    Browse the repository at this point in the history

  6. WIP: Refactor train

    @kks32
    kks32 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    50fe084 View commit details
    Browse the repository at this point in the history

  7. Fix validation dataloader

    @kks32
    kks32 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    27919d1 View commit details
    Browse the repository at this point in the history

  8. Prepare data function

    @kks32
    kks32 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    985a68a View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. check scripts

    Sikan Li committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    0e79137 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'dataloader' of https://github.com/geoelements/gns into … 

    …dataloader
    Sikan Li committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    7748b64 View commit details
    Browse the repository at this point in the history

  3. remove extra file

    Sikan Li committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    1a8546c View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. use python to launch

    Sikan Li committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    7f31242 View commit details
    Browse the repository at this point in the history

  2. black

    Sikan Li committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    2bcd131 View commit details
    Browse the repository at this point in the history

  3. update test

    Sikan Li committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    86b4dbb View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. fix loading simulator and resuming from middle of epoch

    Sikan Li committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    f72b172 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2024

  1. remove n_gpus

    Sikan Li committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    8f4e92c View commit details
    Browse the repository at this point in the history