Skip to content

Commit

Permalink
Merge branch 'dataloader' of https://github.com/geoelements/gns into …
Browse files Browse the repository at this point in the history
…dataloader
  • Loading branch information
Sikan Li committed Jul 2, 2024
2 parents 0e79137 + 985a68a commit 7748b64
Show file tree
Hide file tree
Showing 7 changed files with 191 additions and 516 deletions.
44 changes: 44 additions & 0 deletions .github/workflows/container-gpu.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Build and Push GPU Image to GHCR

on:
push:
paths:
- Dockerfile-GPU
- requirements.txt

env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}

jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
file: ./Dockerfile-GPU
push: true
tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:gpu
labels: ${{ steps.meta.outputs.labels }}
2 changes: 0 additions & 2 deletions Dockerfile-GPU
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,4 @@ RUN pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_co
RUN pip install absl-py autopep8 numpy==1.23.1 dm-tree matplotlib pyevtk pytest tqdm toml
RUN pip install -r requirements.txt



CMD ["/bin/bash"]
44 changes: 10 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Graph Network Simulator (GNS) and MeshNet

[![DOI](https://zenodo.org/badge/427487727.svg)](https://zenodo.org/badge/latestdoi/427487727)
[![CircleCI](https://dl.circleci.com/status-badge/img/gh/geoelements/gns/tree/main.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/gh/geoelements/gns/tree/main)
[![Docker](https://quay.io/repository/geoelements/gns/status "Docker Repository on Quay")](https://quay.io/repository/geoelements/gns)
[![GitHub Actions](https://github.com/geoelements/gns/actions/workflows/train.yml/badge.svg)](https://github.com/geoelements/gns/actions/workflows/train.yml)
[![Docker](https://img.shields.io/badge/container-gpu-limegreen.svg)](https://ghcr.io/geoelements/gns:gpu)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://raw.githubusercontent.com/geoelements/gns/main/license.md)

> Krishna Kumar, The University of Texas at Austin.
Expand Down Expand Up @@ -227,53 +227,29 @@ The dataset is shared on [DesignSafe DataDepot](https://doi.org/10.17603/ds2-fzg
GNS uses [pytorch geometric](https://www.pyg.org/) and [CUDA](https://developer.nvidia.com/cuda-downloads). These packages have specific requirements, please see [PyG installation]((https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html) for details.
> CPU-only installation on Linux (Conda)
> CPU-only installation on Linux/MacOS
```shell
conda install -y pytorch torchvision torchaudio cpuonly -c pytorch
conda install -y pyg -c pyg
conda install -y pytorch-cluster -c pyg
conda install -y absl-py -c anaconda
conda install -y numpy dm-tree matplotlib-base pyevtk -c conda-forge
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip3 install torch_geometric
pip3 install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.3.0+cpu.html
pip3 install -r requirements.txt
```
You can use the [WaterDropletSample](https://github.com/geoelements/gns-sample) dataset to check if your `gns` code is working correctly.

To test the code you can run:

```
pytest test/
```

To test on the small waterdroplet sample:

```
git clone https://github.com/geoelements/gns-sample
TMP_DIR="./gns-sample"
DATASET_NAME="WaterDropSample"
mkdir -p ${TMP_DIR}/${DATASET_NAME}/models/
mkdir -p ${TMP_DIR}/${DATASET_NAME}/rollout/
DATA_PATH="${TMP_DIR}/${DATASET_NAME}/dataset/"
MODEL_PATH="${TMP_DIR}/${DATASET_NAME}/models/"
ROLLOUT_PATH="${TMP_DIR}/${DATASET_NAME}/rollout/"
python -m gns.train --data_path=${DATA_PATH} --model_path=${MODEL_PATH} --ntraining_steps=10
```

### Build Docker Image

Dockerfile-GPU is supplied to build image with GPU support.


### Building GNS on MacOS
```shell
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip3 install torch_geometric
pip3 install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.3.0+cpu.html
pip3 install -r requirements.txt
```
docker pull ghcr.io/geoelements/gns:gpu
```


## GNS training in parallel
GNS can be trained in parallel on multiple nodes with multiple GPUs.
Expand Down
Loading

0 comments on commit 7748b64

Please sign in to comment.