Mixture models are well-established learning approaches that, in computer vision, have mostly been applied to inverse or ill-defined problems. However, they are general-purpose divide-and-conquer techniques, splitting the input space into relatively homogeneous subsets in a data-driven manner. Not only ill-defined but also well-defined complex problems should benefit from them. To this end, we devise a framework for spatial regression using mixture density networks. We realize the framework for object detection and human pose estimation. For both tasks, a mixture model yields higher accuracy and divides the input space into interpretable modes. For object detection, mixture components focus on object scale, with the distribution of components closely following that of ground truth the object scale. This practically alleviates the need for multi-scale testing, providing a superior speed-accuracy trade-off. For human pose estimation, a mixture model divides the data based on viewpoint and uncertainty -- namely, front and back views, with back view imposing higher uncertainty. We conduct experiments on the MS COCO dataset and do not face any mode collapse.
For questions, please contact me at [email protected].
1- Fiest use mixturedense.yml to reproduce the exact Anaconda environment that we have used for our experiments:
conda env create -f mixturedense.yml
To activate the environment:
source activate mixturedense
2- Install COCOAPI
3- Compile deformable convolutional conda env create -f environment.yml(from DCNv2).
cd src/lib/models/networks/DCNv2
./make.sh
To train models from scratch, first, organize the dataset in the following order and see sample commands at experiments
For training and evaluation download the MS COCO dataset and organize it as indicated below:
- Download the images (2017 Train, 2017 Val, 2017 Test) from coco website.
- Download annotation files (2017 train/val and test image info) from coco website.
- Place the data (or create symlinks) to make the data folder like:
optdata_dir
|-- coco
|-- annotations
| |-- instances_train2017.json
| |-- instances_val2017.json
| |-- person_keypoints_train2017.json
| |-- person_keypoints_val2017.json
| |-- image_info_test-dev2017.json
|-- train2017
|-- val2017
|-- test2017ll
To test the models for detction and pose estimation on a images (stored in a directory) use the inference_ctdet.py and inference_pose.py scripts, respectively
Detection_DLA34_CenterNet_COCO_10percent_Train_Resolution_768_100_epochs
Detection_DLA34_MDN_3_COCO_10percent_Train_Resolution_768_100_epochs
This repo is released under the MIT License. We have forked parts of the codebase from other repositories; please refer to the corresponding repositories for their licenses' details.
@article{varamesh2019mixture,
title={Mixture Dense Regression for Object Detection and Human Pose Estimation},
author={Varamesh, Ali and Tuytelaars, Tinne},
journal={arXiv preprint arXiv:1912.00821},
year={2019}
}
Our repo is forked from the amazing codebase of the Object as Points paper