Skip to content

Latest commit

 

History

History
118 lines (87 loc) · 8.91 KB

README.md

File metadata and controls

118 lines (87 loc) · 8.91 KB
Raw

PWC PWC

VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking (CVPR 2023)

This is the official implementation of VoxelNeXt (CVPR 2023). VoxelNeXt is a clean, simple, and fully-sparse 3D object detector. The core idea is to predict objects directly upon sparse voxel features. No sparse-to-dense conversion, anchors, or center proxies are needed anymore. For more details, please refer to:

VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking [Paper]
Yukang Chen, Jianhui Liu, Xiangyu Zhang, Xiaojuan Qi, Jiaya Jia

News

Experimental results

nuScenes Detection Set mAP NDS Download
VoxelNeXt val 60.5 66.6 Pre-trained
VoxelNeXt test 64.5 70.0 Submission
+double-flip test 66.2 71.4 Submission
nuScenes Tracking Set AMOTA AMOTP Download
VoxelNeXt val 70.2 64.0 Results
VoxelNeXt test 69.5 56.8 Submission
+double-flip test 71.0 51.1 Submission
Argoverse2 mAP Download
VoxelNeXt 30.5 Pre-trained
Waymo Vec_L1 Vec_L2 Ped_L1 Ped_L2 Cyc_L1 Cyc_L2
VoxelNeXt-2D 77.94/77.47 69.68/69.25 80.24/73.47 72.23/65.88 73.33/72.20 70.66/69.56
VoxelNeXt-K3 78.16/77.70 69.86/69.42 81.47/76.30 73.48/68.63 76.06/74.90 73.29/72.18
  • We cannot release the pre-trained models of VoxelNeXt on Waymo dataset due to the license of WOD.
  • For Waymo dataset, VoxelNeXt-K3 is an enhanced version of VoxelNeXt with larger model size.
  • During inference, VoxelNeXt can work either with sparse-max-pooling or NMS post-processing. Please install our implemented spconv-plus, if you want to use the sparse-max-pooling inference. Otherwise, please use NMS post-processing by default.

Getting Started

Installation

a. Clone this repository

https://github.com/dvlab-research/VoxelNeXt && cd VoxelNeXt

b. Install the environment

Following the install documents for OpenPCDet.

c. Prepare the datasets.

For nuScenes, Waymo, and Argoverse2 datasets, please follow the document in OpenPCDet.

Evaluation

We provide the trained weight file so you can just run with that. You can also use the model you trained.

cd tools 
bash scripts/dist_test.sh NUM_GPUS --cfg_file PATH_TO_CONFIG_FILE --ckpt PATH_TO_MODEL
#For example,
bash scripts/dist_test.sh 8 --cfg_file PATH_TO_CONFIG_FILE --ckpt PATH_TO_MODEL

Training

bash scripts/dist_train.sh NUM_GPUS --cfg_file PATH_TO_CONFIG_FILE
#For example,
bash scripts/dist_train.sh 8 --cfg_file PATH_TO_CONFIG_FILE

Citation

If you find this project useful in your research, please consider citing:

@inproceedings{chen2023voxenext,
  title={VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking},
  author={Yukang Chen and Jianhui Liu and Xiangyu Zhang and Xiaojuan Qi and Jiaya Jia},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2023}
}

An introduction video on YouTube can be found here. IMAGE ALT TEXT

Acknowledgement

  • This work is built upon the OpenPCDet and spconv.
  • This work is motivated by FSD. And we follow FSD for the Argoverse2 data processing.

Our Works in LiDAR-based Autonumous Driving

  • VoxelNeXt (CVPR 2023) [Paper] [Code] Fully Sparse VoxelNet for 3D Object Detection and Tracking.
  • Focal Sparse Conv (CVPR 2022 Oral) [Paper] [Code] Dynamic sparse convolution for high performance.
  • Spatial Pruned Conv (NeurIPS 2022) [Paper] [Code] 50% FLOPs saving for efficient 3D object detection.
  • LargeKernel3D (CVPR 2023) [Paper] [Code] Large-kernel 3D sparse CNN backbone.
  • SphereFormer (CVPR 2023) [Paper] [Code] Spherical window 3D transformer backbone.
  • spconv-plus A library where we combine our works into spconv.
  • SparseTransformer A library that includes high-efficiency transformer implementations for sparse point cloud or voxel data.

License

This project is released under the Apache 2.0 license.