Volumetric Rendering with Baked Quadrature Fields

Authors: Gopal Sharma, Daniel Rebain, Kwang Moo Yi and Andrea Tagliasacchi

This repository contains codebase for the Quadfield paper published at ECCV-2024.

Paper | Website

Installation

Experiments are done using Python 3.10, cuda 11.3 and torch 1.12.0. Further see the requirements in requirements.txt. Additional packages required for baking the texture maps: xatlas and segmentor for mesh segmentation.

Dataset

Download NeRF synthetic and Shelly datasets and put the scene directories in dataset directory.

├── dataset
│   ├── chair
│   └── kahdy
│   └── lego

Experiments

Experiments are done on Nvidia 3090 GPU.

Training on NeRF synthetic

Train Nerf on a scene.

bash scripts/run_nerfsynthetic.sh

Train quadrature field on a scene.

bash scripts/run_nerfsynthetic_field.sh

Extract meshes.

bash scripts/run_nerfsynthetic_mc.sh

Finetune field.

bash scripts/run_nerfsynthetic_finetune.sh

The following steps are only needed for baking:

Fit spherical gaussians.

bash scripts/run_nerfsynthetic_fit_sg.sh

Bake neural features and evaluate the model.

bash scripts/run_nerfsynthetic_baking.sh

Training on Shelly dataset

Train Nerf on a scene.

bash scripts/run_shelly.sh

Train quadrature field on a scene.

bash scripts/run_shelly_field.sh

Extract meshes.

bash scripts/run_shelly_mc.sh

Finetune field.

bash scripts/run_shelly_finetune.sh

The following steps are only needed for baking:

Fit spherical gaussians.

bash scripts/run_shelly_fit_sg.sh

Bake neural features and evaluate the model.

bash scripts/run_shelly_baking.sh

Acknowledgements

We thank Ruilong Li for releasing nerfacc. Our codebase is heavily built on nerfacc and pycolmap.

Bibliography

Please find below the BibTeX entry for this project:

@article{quadfields,
  title     = {Volumetric Rendering with Baked Quadrature Fields},
  author    = {Gopal Sharma, Daniel Rebain, Andrea Tagliasacchi, Kwang Moo Yi},
  journal   = {ECCV},
  year      = {2024},
}