Building & Training an Image-to-Image Generative Model

Slides for the workshop are here.

The code on this repo is an adaptation of the Pytorch pix2pix implementation (repo) and the Tensorflow implementation of HED detector (repo).

The models implemented are based on:

• Isola, et al. 2016. Image-to-Image Translation with Conditional Adversarial Networks. arxiv
• Xie, et al. 2015. Holistically-Nested HED Detection. arxiv

Data

The data used to train the models can be found here.

Quick Commands

Resize a folder of images

First edit the input and output folders in /dataset_creation/reduce_folder.sh.

sh reduce_folder.sh

Extract Canny Edges from a folder

cd dataset_creation
python2 auto_canny.py -i [input folder] -o [output folder]

Extract HED Edges from a folder

First edit the input and output folders in [/holy-edge/hed/config/hed.yaml]. Create a list of the files from which to extract the edges outside the folder:

cd [image_folder]
cd ..
touch test.lst
ls [image_folder] > test.lst

And then run the HED detector

cd holy-edge
CUDA_VISIBLE_DEVICES=0 python2 run-hed --test --config-file ./hed/config/hed.yaml

Generate images with a pre-trained Image-to-Image model

Edit the necessary parameters in /predict.py

python2 predict.py --exp_name [experiment_name]

Train an Image-to-Image Model

python2 train.py --exp_name [experiment_name] --dataroot_faces [source images folder] --dataroot_edges [edges image folder]

Training results and model checkpoints will be saved in runs/experiment_name

Experiments

CelebA - centered and cropped

---

No adversarial loss

Canny edges - Lambda 1e-4

Train Set

Test Set

Canny edges - Lambda 1e-3

Train Set

Test Set

HED edges - Lambda 5e-4

Train Set

Test Set

HED edges - Lambda 1e-1

Train Set

Test Set

HED edges - Lambda 1e-2

Test Set

CelebA - faces in the wild

---

HED edges - Lambda 1e-3

Train Set

Test Set