|
1 | 1 | # PointPWC
|
2 |
| -PointPWC-Net is a deep coarse-to-fine network designed for 3D scene flow estimation from 3D point clouds. |
| 2 | +This is the code for [PointPWC-Net](https://arxiv.org/abs/1911.12408), a deep coarse-to-fine network designed for 3D scene flow estimation from 3D point clouds. |
| 3 | + |
| 4 | +## Prerequisities |
| 5 | +Our model is trained and tested under: |
| 6 | +* Python 3.6.9 |
| 7 | +* NVIDIA GPU + CUDA CuDNN |
| 8 | +* PyTorch (torch == 1.5) |
| 9 | +* scipy |
| 10 | +* tqdm |
| 11 | +* sklearn |
| 12 | +* numba |
| 13 | +* cffi |
| 14 | + |
| 15 | +Compile the furthest point sampling, grouping and gathering operation for PyTorch. We use the operation from this [repo](https://github.com/sshaoshuai/Pointnet2.PyTorch). |
| 16 | + |
| 17 | +```shell |
| 18 | +cd pointnet2 |
| 19 | +python setup.py install |
| 20 | +cd ../ |
| 21 | +``` |
| 22 | + |
| 23 | +## Data preprocess |
| 24 | + |
| 25 | +For fair comparison with previous methods, we adopt the preprocessing steps in [HPLFlowNet](https://web.cs.ucdavis.edu/~yjlee/projects/cvpr2019-HPLFlowNet.pdf). Please refer to [repo](https://github.com/laoreja/HPLFlowNet). We alos copy the preprocessing instructions here for your reference. |
| 26 | + |
| 27 | +* FlyingThings3D: |
| 28 | +Download and unzip the "Disparity", "Disparity Occlusions", "Disparity change", "Optical flow", "Flow Occlusions" for DispNet/FlowNet2.0 dataset subsets from the [FlyingThings3D website](https://lmb.informatik.uni-freiburg.de/resources/datasets/SceneFlowDatasets.en.html) (we used the paths from [this file](https://lmb.informatik.uni-freiburg.de/data/FlyingThings3D_subset/FlyingThings3D_subset_all_download_paths.txt), now they added torrent downloads) |
| 29 | +. They will be upzipped into the same directory, `RAW_DATA_PATH`. Then run the following script for 3D reconstruction: |
| 30 | + |
| 31 | +```bash |
| 32 | +python3 data_preprocess/process_flyingthings3d_subset.py --raw_data_path RAW_DATA_PATH --save_path SAVE_PATH/FlyingThings3D_subset_processed_35m --only_save_near_pts |
| 33 | +``` |
| 34 | + |
| 35 | +* KITTI Scene Flow 2015 |
| 36 | +Download and unzip [KITTI Scene Flow Evaluation 2015](http://www.cvlibs.net/download.php?file=data_scene_flow.zip) to directory `RAW_DATA_PATH`. |
| 37 | +Run the following script for 3D reconstruction: |
| 38 | + |
| 39 | +```bash |
| 40 | +python3 data_preprocess/process_kitti.py RAW_DATA_PATH SAVE_PATH/KITTI_processed_occ_final |
| 41 | +``` |
| 42 | + |
| 43 | +## Get started |
| 44 | + |
| 45 | +Here are some demo results: |
| 46 | + |
| 47 | +<img src="./images/FlyingThings3D.gif"> |
| 48 | + |
| 49 | +<img src="./images/Kitti.gif"> |
| 50 | + |
| 51 | +### Train |
| 52 | +Set `data_root` in the configuration file to `SAVE_PATH` in the data preprocess section. Then run |
| 53 | +```bash |
| 54 | +python3 train.py config_train.yaml |
| 55 | +``` |
| 56 | + |
| 57 | +### Evaluate |
| 58 | +Set `data_root` in the configuration file to `SAVE_PATH` in the data preprocess section. Then run |
| 59 | +```bash |
| 60 | +python3 evaluate.py config_evaluate.yaml |
| 61 | +``` |
| 62 | + |
| 63 | +We upload one pretrained model in ```pretrain_weights```. |
| 64 | + |
| 65 | +## Citation |
| 66 | + |
| 67 | +If you use this code for your research, please cite our paper. |
| 68 | + |
| 69 | +``` |
| 70 | +@article{wu2019pointpwc, |
| 71 | + title={PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Clouds}, |
| 72 | + author={Wu, Wenxuan and Wang, Zhiyuan and Li, Zhuwen and Liu, Wei and Fuxin, Li}, |
| 73 | + journal={arXiv preprint arXiv:1911.12408}, |
| 74 | + year={2019} |
| 75 | +} |
| 76 | +``` |
| 77 | + |
| 78 | +## Acknowledgement |
| 79 | + |
| 80 | +We thank [repo](https://github.com/sshaoshuai/Pointnet2.PyTorch) and [repo](https://github.com/laoreja/HPLFlowNet) for subsampling, grouping and data preprocessing related functions. |
3 | 81 |
|
4 | 82 |
|
5 |
| -scipy |
6 |
| -tqdm |
7 |
| -sklearn |
8 |
| -pptk |
9 |
| -numba |
10 |
| -cffi |
|
0 commit comments