GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis

This repository is the official PyTorch implementation of the paper:

   GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis
Yiqing Liang‡, Numair Khan, Zhengqin Li, Thu Nguyen-Phuoc, Douglas Lanman, James Tompkin‡, Lei Xiao

        ‡

   WACV, 2025

   Paper / Arxiv

Getting Started

This code has been developed with Anaconda (Python 3.9), CUDA 12.1.1 on Red Hat Enterprise Linux 9.2, one NVIDIA GeForce RTX 3090 GPU.
Based on a fresh Anaconda environment gaufre, following packages need to be installed:

conda create -p [YourPath]/gaufre python=3.9
conda activate [YourPath]/gaufre
conda install -c anaconda libstdcxx-ng
conda install -c menpo opencv 
conda install -c conda-forge plyfile==0.8.1
pip install tqdm imageio

pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu121
python -c "import torch; print(torch.cuda.is_available())" # verify that torch is installed correctly

pip install matplotlib
pip install torchmetrics
pip install requests 
pip install plotly
pip install dash
pip install scikit-learn
pip install yaml
pip install tensorboard 
pip install scipy
pip install kornia
pip install lpips

# install from local folders 
cd submodules/dqtorch
python setup.py install
cd ../..
pip install submodules/depth-diff-gaussian-rasterization
pip install submodules/simple-knn

Dataset

We follow the data organization of "Monocular Dynamic Gaussian Splatting is Fast and Brittle but Smooth Motion Helps", which could be downloaded here.

To use, one needs to unzip each [NestedPath]/[Scene].zip to be folder [NestedPath]/[Scene].

Training and Inference

To train GauFRe on a scene [NestedPath]/[Scene], and save output to folder [OutputPath],

conda activate [YourPath]/gaufre
# for real-world scenes
bash scripts/trainval_real.sh [NestedPath]/[Scene] [OutputPath]
# for synthetic scenes
bash scripts/trainval_synthetic.sh [NestedPath]/[Scene] [OutputPath]

Acknowledgement

Please cite our paper if you found our work useful:

@inproceedings{liang2024gaufre,  
  Author = {Liang, Yiqing and Khan, Numair and Li, Zhengqin and Nguyen-Phuoc, Thu and Lanman, Douglas and Tompkin, James and Xiao, Lei},  
  Booktitle = {WACV},  
  Title = {GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis},  
  Year = {2025}  
}

• We thank https://github.com/graphdeco-inria/gaussian-splatting for source code reference.