Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Prior

Project page | Paper | Data

This is the official repo for the implementation of RENI: A Rotation-Equivariant Natural Illumination Model.

If you use our code, please cite the following paper:

@inproceedings{
  gardner2022rotationequivariant,
  title={Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Prior},
  author={James A D Gardner and Bernhard Egger and William A P Smith},
  booktitle={Advances in Neural Information Processing Systems},
  editor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},
  year={2022},
  url={https://openreview.net/forum?id=cj6K4IWVomU}
}

News:

27/11/2023: Checkout our improved model RENI++: A Rotation-Equivariant, Scale-Invariant, Natural Illumination Prior here

01/11/2022: Updated code! Now implemented using PyTorch-Lightning. Refactored code makes training and using RENI in downstream tasks easier.

15/09/2022: Accepted to NeurIPS 2022!!!

Setup:

1. Clone this repository:

git clone https://github.com/JADGardner/RENI.git

2. Setup conda environment:

cd RENI
conda env create -f environment.yml
conda activate reni

4. You can download the RENI dataset and pre-trained models using setup.py

python setup.py

5. You can train a RENI from scratch by setting the hyperparameters in configs/experiment.yaml and running the run.py script:

python run.py --cfg_path configs/experiment.yaml --gpus '0, 1, 2, 3'

6. The example notebook demonstrates using a pre-trained RENI in a downstream task as a prior for environment map in-painting.