forked from Gorilla-Lab-SCUT/frustum-convnet
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
35a0899
commit d5a56de
Showing
63 changed files
with
87,237 additions
and
14 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,20 +1,143 @@ | ||
## Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection | ||
# Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection | ||
|
||
This respository is the code for our IROS 2019 paper [[arXiv]](https://arxiv.org/abs/1903.01864). | ||
## Code | ||
The code will be released within three weeks. | ||
This repository is the code for our IROS 2019 paper [[arXiv]](https://arxiv.org/abs/1903.01864). | ||
|
||
## Citation | ||
|
||
If you find our work useful, please consider citing. | ||
``` | ||
@inproceedings{wang2019frustum, | ||
title={Frustum ConvNet: Sliding Frustums to | ||
Aggregate Local Point-Wise Features for Amodal 3D | ||
Object Detection}, | ||
author={Wang, Zhixin and Jia, Kui}, | ||
booktitle={IROS}, | ||
year={2019}, | ||
organization={IEEE} | ||
If you find this work useful in your research, please consider citing. | ||
|
||
```BibTeX | ||
@inproceedings{wang2019frustum, | ||
title={Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection}, | ||
author={Wang, Zhixin and Jia, Kui}, | ||
booktitle={IROS}, | ||
year={2019}, | ||
organization={IEEE} | ||
} | ||
``` | ||
|
||
## Installation | ||
|
||
### Requirements | ||
|
||
* PyTorch 1.0+ | ||
* Python 3.6+ | ||
|
||
We test our code under Ubuntu-16.04 with CUDA-9.0, CUDNN-7.0, Python-3.7.2, PyTorch-1.0. | ||
|
||
### Clone the repository and install dependencies | ||
|
||
```shell | ||
git clone https://github.com/zhixinwang/frustum-convnet.git | ||
``` | ||
|
||
You may need to install extra packages, like pybind11, opencv, yaml, tensorflow(optional). | ||
|
||
If you want to use tensorboard to visualize the training status, you should install tensorflow (CPU version is enough). | ||
Otherwise, you should set the config 'USE_TFBOARD: False' in cfgs/\*.yaml. | ||
|
||
### Compile extension | ||
|
||
```shell | ||
cd ops | ||
bash clean.sh | ||
bash make.sh | ||
``` | ||
|
||
## Download data | ||
|
||
Download the KITTI 3D object detection dataset from [here](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d) and organize them as follows. | ||
|
||
```text | ||
data/kitti | ||
├── testing | ||
│ ├── calib | ||
│ ├── image_2 | ||
│ └── velodyne | ||
└── training | ||
├── calib | ||
├── image_2 | ||
├── label_2 | ||
└── velodyne | ||
``` | ||
|
||
## Training and evaluation | ||
|
||
### First stage | ||
|
||
Run following command to prepare pickle files for car training. We use the 2D detection results from F-PointNets. | ||
The pickle files will be saved in `kitti/data/pickle_data`. | ||
|
||
```shell | ||
python kitti/prepare_data.py --car_only --gen_train --gen_val --gen_val_rgb_detection | ||
``` | ||
|
||
Run following commands to train and evaluate the final model. You can use `export CUDA_VISIBLE_DEVICES=\?` to specify which GPU to use. | ||
And you can modify the setting after `OUTPUT_DIR` to set a directory to save the log, model files and evaluation results. All the config settings are under the configs/config.py. | ||
|
||
```shell | ||
python train/train_net_det.py --cfg cfgs/det_sample.yaml OUTPUT_DIR output/car_train | ||
python train/test_net_det.py --cfg cfgs/det_sample.yaml OUTPUT_DIR output/car_train TEST.WEIGHTS output/car_train/model_0050.pth | ||
``` | ||
|
||
We also provide the shell script, so you can also run `bash scripts/car_train.sh` instead. | ||
|
||
### Refinement stage | ||
|
||
Run following command to prepare pickle files for car training. We use the first stage predicted results. If you don't use the default directory in first stage, you should change the corresponding directory in [here](kitti/prepare_data_refine.py#L911) and [here](kitti/prepare_data_refine.py#L895) before running following commands. The pickle files will be saved in `kitti/data/pickle_data_refine`. | ||
|
||
```shell | ||
python kitti/prepare_data_refine.py --car_only --gen_train --gen_val_det --gen_val_rgb_detection | ||
``` | ||
|
||
Run following commands to train and evaluate the final model. | ||
|
||
```shell | ||
python train/train_net_det.py --cfg cfgs/refine_car.yaml OUTPUT_DIR output/car_train_refine | ||
python train/test_net_det.py --cfg cfgs/refine_car.yaml OUTPUT_DIR output/car_train_refine TEST.WEIGHTS output/car_train_refine/model_0050.pth | ||
``` | ||
|
||
We also provide the shell script, so you can also run `bash scripts/car_train_refine.sh` instead. | ||
|
||
### All commands in one script file | ||
|
||
You can simply run `bash scripts/car_all.sh` to execute all the above commands. | ||
|
||
## Pretrained models | ||
We provide the pretrained models for car category, you can download from [here](https://drive.google.com/open?id=1z7bBVOjtJx6qW0oKP1EcQxECqq0HP3_9). | ||
After extracting the files under root directory, you can run `bash scripts/eval_pretrained_models.sh` to evaluate the pretrained models. | ||
The performance on validation set is as follows: | ||
|
||
```text | ||
# first stage | ||
Car [email protected], 0.70, 0.70: | ||
bbox AP:98.33, 90.40, 88.24 | ||
bev AP:90.32, 88.02, 79.41 | ||
3d AP:87.76, 77.41, 68.79 | ||
# refinement stage | ||
Car [email protected], 0.70, 0.70: | ||
bbox AP:98.43, 90.39, 88.15 | ||
bev AP:90.42, 88.99, 86.88 | ||
3d AP:89.31, 79.08, 77.17 | ||
``` | ||
|
||
## Note | ||
|
||
Since we update our code from PyTorch-0.3.1 to PyTorch-1.0 and our code uses may random sample operations, the results may be not exactly the same as these reported in our paper. | ||
But the difference should be +-0.5\%, if you can not get the similar results, please contact me. I am still working to make results stable. | ||
|
||
Our code is supported multiple GPUs for training, but now the training is very fast for small dataset, like KITTI, SUN-RGBD. All the steps will finish in one day on single GPU. | ||
|
||
## TODO | ||
- [ ] provide a demo script to visualize the detection resutls | ||
- [ ] add the support of SUN-RGBD dataset | ||
|
||
## Acknowledgements | ||
|
||
Part of the code was adapted from [F-PointNets](https://github.com/charlesq34/frustum-pointnets). | ||
|
||
## License | ||
|
||
Our code is released under [MIT license](LICENSE). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
TRAIN: | ||
BATCH_SIZE: 32 | ||
START_EPOCH: 0 | ||
MAX_EPOCH: 50 | ||
OPTIMIZER: adam | ||
BASE_LR: 0.001 | ||
MIN_LR: 1e-5 | ||
GAMMA: 0.1 | ||
LR_STEPS: [20] | ||
MOMENTUM: 0.9 | ||
WEIGHT_DECAY: 0.0001 | ||
|
||
MODEL: | ||
FILE: models/det_base.py | ||
NUM_CLASSES: 2 | ||
|
||
TEST: | ||
BATCH_SIZE: 32 | ||
METHOD: nms | ||
|
||
DATA: | ||
FILE: datasets/provider_sample.py | ||
DATA_ROOT: kitti/data/pickle_data | ||
CAR_ONLY: True | ||
RTC: True | ||
WITH_EXTRA_FEAT: False | ||
NUM_SAMPLES: 1024 | ||
STRIDE: (0.25, 0.5, 1.0, 2.0) | ||
HEIGHT_HALF: (0.25, 0.5, 1.0, 2.0) | ||
|
||
RESUME: False | ||
NUM_GPUS: 1 | ||
OUTPUT_DIR: 'output/car_train' | ||
SAVE_SUB_DIR: 'val_nms' | ||
USE_TFBOARD: True | ||
NUM_WORKERS: 4 | ||
FROM_RGB_DET: True | ||
disp: 100 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
TRAIN: | ||
BATCH_SIZE: 32 | ||
START_EPOCH: 0 | ||
MAX_EPOCH: 50 | ||
OPTIMIZER: adam | ||
BASE_LR: 0.001 | ||
MIN_LR: 1e-5 | ||
GAMMA: 0.1 | ||
LR_STEPS: [20] | ||
MOMENTUM: 0.9 | ||
WEIGHT_DECAY: 0.0001 | ||
|
||
MODEL: | ||
FILE: models/det_base.py | ||
NUM_CLASSES: 2 | ||
|
||
TEST: | ||
BATCH_SIZE: 32 | ||
METHOD: nms | ||
|
||
DATA: | ||
FILE: datasets/provider_sample_refine.py | ||
DATA_ROOT: kitti/data/pickle_data_refine | ||
CAR_ONLY: True | ||
RTC: True | ||
WITH_EXTRA_FEAT: False | ||
NUM_SAMPLES: 512 | ||
STRIDE: (0.1, 0.2, 0.4, 0.8) | ||
HEIGHT_HALF: (0.1, 0.2, 0.4, 0.8) | ||
EXTEND_FROM_DET: False | ||
|
||
|
||
RESUME: False | ||
NUM_GPUS: 1 | ||
OUTPUT_DIR: 'output/car_train_refine' | ||
SAVE_SUB_DIR: 'val_nms' | ||
USE_TFBOARD: True | ||
NUM_WORKERS: 4 | ||
FROM_RGB_DET: True | ||
disp: 100 |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,65 @@ | ||
# Copyright (c) 2017-present, Facebook, Inc. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
############################################################################## | ||
|
||
"""A simple attribute dictionary used for representing configuration options.""" | ||
|
||
from __future__ import absolute_import | ||
from __future__ import division | ||
from __future__ import print_function | ||
from __future__ import unicode_literals | ||
|
||
|
||
class AttrDict(dict): | ||
IMMUTABLE = '__immutable__' | ||
|
||
def __init__(self, *args, **kwargs): | ||
super(AttrDict, self).__init__(*args, **kwargs) | ||
self.__dict__[AttrDict.IMMUTABLE] = False | ||
|
||
def __getattr__(self, name): | ||
if name in self.__dict__: | ||
return self.__dict__[name] | ||
elif name in self: | ||
return self[name] | ||
else: | ||
raise AttributeError(name) | ||
|
||
def __setattr__(self, name, value): | ||
if not self.__dict__[AttrDict.IMMUTABLE]: | ||
if name in self.__dict__: | ||
self.__dict__[name] = value | ||
else: | ||
self[name] = value | ||
else: | ||
raise AttributeError( | ||
'Attempted to set "{}" to "{}", but AttrDict is immutable'. | ||
format(name, value) | ||
) | ||
|
||
def immutable(self, is_immutable): | ||
"""Set immutability to is_immutable and recursively apply the setting | ||
to all nested AttrDicts. | ||
""" | ||
self.__dict__[AttrDict.IMMUTABLE] = is_immutable | ||
# Recursively set immutable state | ||
for v in self.__dict__.values(): | ||
if isinstance(v, AttrDict): | ||
v.immutable(is_immutable) | ||
for v in self.values(): | ||
if isinstance(v, AttrDict): | ||
v.immutable(is_immutable) | ||
|
||
def is_immutable(self): | ||
return self.__dict__[AttrDict.IMMUTABLE] |
Oops, something went wrong.