Skip to content

Commit

Permalink
update links
Browse files Browse the repository at this point in the history
  • Loading branch information
zhenjia-xu committed Oct 10, 2020
1 parent a20dffd commit a5959a4
Show file tree
Hide file tree
Showing 5 changed files with 35 additions and 11 deletions.
23 changes: 12 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Learning 3D Dynamic Scene Representations for Robot Manipulation
### [project](https://dsr-net.cs.columbia.edu/) | [paper(TBD)]() | [videos(TBD)]()
### [project](https://dsr-net.cs.columbia.edu/)
## Overview
This repo contains the PyTorch implementation for paper Learning 3D Dynamic Scene Representations for Robot Manipulation.
![teaser](figures/teaser.jpg)

## Content

Expand All @@ -16,26 +18,25 @@ The code is built with Python 3.6. Libraries are listed in [requirements.txt](re
## Data Preparation

### Download Testing Data
The following two testing datasets can be download in [link(TBD)]().
- Sim: 400 sequences, generated in pybullet.
- Real: 150 sequences, with full annotations.
The following two testing datasets can be download.
- [Sim](https://dsr-net.cs.columbia.edu/download/data/sim_test_data.zip): 400 sequences, generated in pybullet.
- [Real](https://dsr-net.cs.columbia.edu/download/data/real_test_data.zip): 150 sequences, with full annotations.

### Generate Training Data

Download object mesh from [link(TBD)]().
Download object mesh: [shapenet](https://dsr-net.cs.columbia.edu/download/object_models/shapenet.zip) and [ycb](https://dsr-net.cs.columbia.edu/download/object_models/ycb.zip).

To generate data in simulation, one can run
```
python data_generation.py --data_path [path to data] --train_num [number of training sequences] --train_num [number of testing sequences] --object_type [type of objects]
python data_generation.py --data_path [path to data] --train_num [number of training sequences] --test_num [number of testing sequences] --object_type [type of objects]
```
Where the `object_type` can be `cube`, `shpenet`, or `ycb`.
The training data in the paper can be generated with the followint scripts:
```
# cube
python data_generation.py --data_path data/cube_train --train_num 4000 --train_num 400 --object_type cube
python data_generation.py --data_path data/cube_train --train_num 4000 --test_num 400 --object_type cube
# shapenet
python data_generation.py --data_path data/shapenet_train --train_num 4000 --train_num 400 --object_type shapenet
python data_generation.py --data_path data/shapenet_train --train_num 4000 --test_num 400 --object_type shapenet
```

## Pretrained Models
Expand Down Expand Up @@ -100,6 +101,6 @@ python train.py --exp dsr_stage3 --resume [path to stage2] --data_path [path to
```

### Training of Baselines
For `nowarp` and `gtwarp`, use the same scripts as DSR-NEt with corresponding `model_type`.
- `nowarp` and `gtwarp`. Use the same scripts as DSR-Net with corresponding `model_type`.

For `single` and `3dflow`, only two stages is used (one for cube, one for shapenet) and `seq_len` always choose `1`.
- `single` and `3dflow`. Two-stage training: (1) single step on cube dataset; (2) single step on Shapenet dataset.
9 changes: 9 additions & 0 deletions data/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Data

### Generate training data
- Please refer to [link](../README.md).
### Donload testing data
- The following two testing datasets can be download.
- [Sim](https://dsr-net.cs.columbia.edu/download/data/sim_test_data.zip): 400 sequences, generated in pybullet.
- [Real](https://dsr-net.cs.columbia.edu/download/data/real_test_data.zip): 150 sequences, with full annotations.
- Unzip `real_test_data.zip` and `sim_test_data.zip` in this folder.
Binary file added figures/teaser.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 5 additions & 0 deletions object_models/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Object Models

### Object meshes are used for data generation in simulation.
- Download object meshes: [shapenet](https://dsr-net.cs.columbia.edu/download/object_models/shapenet.zip) and [ycb](https://dsr-net.cs.columbia.edu/download/object_models/ycb.zip).
- Unzip `shapenet.zip` and `ycb.zip` in this folder.
9 changes: 9 additions & 0 deletions pretrained_models/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Pretrained Models

### The following pretrained models are provided:
- [dsr](dsr.pth): DSR-Net introduced in the paper. (without real data finetuning)
- [dsr_ft](dsr_ft.pth): DSR-Net introduced in the paper. (with real data finetuning)
- [single](single.pth): It does not use any history aggregation.
- [nowarp](nowarp.pth): It does not warp the representation before aggregation.
- [gtwarp](gtwarp.pth): It warps the representation with ground truth motion (i.e., performance oracle)
- [3dflow](3dflow.pth): It predicts per-voxel scene flow for the entire 3D volume.

0 comments on commit a5959a4

Please sign in to comment.