DeepSphere-Weather - Deep Learning on the sphere for weather/climate applications.

t850_state_obs_vs_pred.mp4

The code in this repository provides a scalable and flexible framework to apply convolutions on spherical unstructured grids for weather/climate applications.

ATTENTION: The code is subject to changes in the coming weeks / months.

The folder experiments (will) provide examples for:

• Weather forecasting using autoregressive CNN models
• Weather field downscaling (aka superesolution) [in preparation].
• Classication of atmospheric features (i.e. tropical cyclone and atmospheric rivers) [in preparation].

The folder tutorials (will) provide jupyter notebooks describing various features of DeepSphere-Weather.

The folder docs (will) contains slides and notebooks explaining the DeepSphere-Weather concept.

Installation

For a local installation, follow the below instructions.

1. Clone this repository.

   git clone https://github.com/deepsphere/deepsphere-weather.git
   cd deepSphere-weather

2. Install manually the following dependencies:

   • Install first pytorch and its extensions on GPU:

     conda install -c conda-forge pytorch-gpu  

   • If you don't have GPU available install it on CPU:

     conda install -c conda-forge pytorch-cpu  

   • Install the required packages:

   conda create --name weather python=3.8
   conda install xarray dask cdo h5py h5netcdf netcdf4 zarr numcodecs rechunker xskillscore
   conda install notebook jupyterlab
   conda install matplotlib-base cartopy pycairo seaborn cycler
   conda install numpy pandas numba scipy bottleneck
   conda install yaml tabulate tqdm deepdiff
   conda install healpy igl shapely      
   pip install git+https://github.com/epfl-lts2/pygsp@sphere-graphs
   pip install torchinfo

   • Clone the required repository:

   git clone [email protected]:ghiggi/xverif.git
   git clone [email protected]:ghiggi/xscaler.git
   git clone [email protected]:ghiggi/xsphere.git
   git clone [email protected]:ghiggi/xforecasting.git

3. Alternatively install the dependencies using one of the appropriate below environment.yml files:

   conda env create -f environment_python3.8.5.yml
   conda env create -f environment_python3.9.yml

   and clone the required repository:

   git clone [email protected]:ghiggi/xverif.git
   git clone [email protected]:ghiggi/xscaler.git
   git clone [email protected]:ghiggi/xsphere.git
   git clone [email protected]:ghiggi/xforecasting.git

4. Add the PYTHONPATH (i.e. in the .bashrc) for the following packages. Here is an example:

    export PYTHONPATH="${PYTHONPATH}:/home/ghiggi/Python_Packages/xscaler"
    export PYTHONPATH="${PYTHONPATH}:/home/ghiggi/Python_Packages/xverif"
    export PYTHONPATH="${PYTHONPATH}:/home/ghiggi/Python_Packages/xsphere"
    export PYTHONPATH="${PYTHONPATH}:/home/ghiggi/Python_Packages/xforecasting"  
    export PYTHONPATH="${PYTHONPATH}:/home/ghiggi/Projects/deepsphere-weather"

Tutorials

• spherical_grids.ipynb: get a taste of samplings/pixelizations of the sphere.
• interpolation_pooling.ipynb: get to understand our generalized pooling based on interpolation between samplings.

Data structure

Here we are going to document the data structure

The data required for model training and evaluation are stored with the following folder hierarchy.

preprocessed
└───ERA5_HRES
|   └─── <sampling>
|         └─── Data
|         └─── Scalers
|         └─── Benchmarks 
|         └─── Climatology 
| 
└───IF5_HRES
|    └─── <sampling>
|         └─── Data
|    
└───IF5_ENS
|   └─── <sampling>
         └─── Data

On the LTE servers, data are available in the directory /ltenas3/data/DeepSphere

The data are available upon request to the authors.

Reproducing our results

Contributors

• Gionata Ghiggi slides video
• Michaël Defferrard
• Wentao Feng [code, slides]
• Yann Yasser Haddad [code, slides]
• Natalie Bolón Brun [code]
• Icíar Lloréns Jover [code, report & slides]

License

The content of this repository is released under the terms of the MIT license.