Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation

Created by He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, Leonidas J. Guibas from Stanford University, Google Inc., Princeton University, Facebook AI Research.

Citation

If you find our work useful in your research, please consider citing:

 @InProceedings{Wang_2019_CVPR,
               author = {Wang, He and Sridhar, Srinath and Huang, Jingwei and Valentin, Julien and Song, Shuran and Guibas, Leonidas J.},
               title = {Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation},
               booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
               month = {June},
               year = {2019}
 }

Introduction

This is a keras and tensorflow implementation of Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation, a CVPR 2019 oral paper.

The repository includes:

• Source code of NOCS.
• Training code
• Detection and evaluation code
• Pre-trained weights
• Example code for using Blender to render NOCS map

For more information, please visit the project page.

Requirements

This code has been tested with

• CUDA 10.0 & cuDNN 7.41
• Python 3.5
• Tensorflow 1.14.0
• Keras 2.3.0

Datasets

• CAMERA Dataset: Training/Test/IKEA_backgrounds/Composed_depths

+ Composed depth images contain the depths of both foreground synthetic objects and background real scenes for all train and validation data

• Real Dataset: Training/Test
• Ground truth pose annotation (for an easier evaluation): Val&Real_test
• Object Meshes

You can download the files and store them under data/.

NOTE: You are required to cite our paper if you use the dataset. The data is only for non-commercial use. Please reach out to us for other use cases.

Pretrain weights

You can find the following checkpoints in this download link:

• NOCS RCNN jointly trained on CAMERA, Real & MS COCO with 32 bin classification setting
• NOCS RCNN jointly trained on CAMERA, Real & MS COCO with regression setting
• Mask RCNN pretrained on MS COCO dataset

You can download the checkpoints and store them under logs/.

Training

# Train a new model from pretrained COCO weight
python3 train.py

Detection and evaluation

# Detect using a checkpoint
python3 detect_eval.py --mode detect --ckpt_path=/logs/ckpt --draw

# Evaluate a checkpoint
python3 detect_eval.py --mode eval --ckpt_path=/output/ckpt 

Rendering NOCS Map using blender 2.79b

# Render the NOCS map for the default cube in Blender
# See images/nocs_map_cube.png for the reference.
blender -b --python nocs_map_cube.py