Super Resolution of picture images using deep learning.
SeRanet upscales picture image size to x2.
* Below sample pictures are obtained from PEXELS, providing public domain photos.
Before upscale
Conventional method: image upscaled with lanczos method (with OpenCV library)
SeRanet: image upscaled using seranet_v1
Original image (= Ground truth data, for reference)
SeRanet project aims to Study and Research how deep convolutional neural network works to learn super resolution of the image.
It is developed with python on chainer framework, flexible machine learning library.
Explanation material about super resolution through convolutional neural network is uploaded on SlideShare.
The project is inspired by following two reference
- Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang, "Image Super-Resolution Using Deep Convolutional Networks", http://arxiv.org/abs/1501.00092
The term "SRCNN", super resolution using deep convolutional neural network, is introduced in this paper.
Image super resolution for Anime-Style art and photo. You can also test the application at http://waifu2x.udp.jp/index.html.
Machine learning library
Machine learning library which can be written in python. It is open source on github
I tested the software with Ubuntu 14.04, python 2.7.
Running the software requires a lot of memory, more than 8GB RAM recommended (it depends on which architecture to use).
-
numpy
-
chainer
Follow github page for installation. Setting up GPU (enable CUDA, cuDNN) accelerates the calculation. It is recommended to use GPU if possible.
- OpenCV
OpenCV is used for image processing. Installation page
Also, please set environment variable PYTHONPATH='.'.
Add below code in ~/.bashrc.
export PYTHONPATH=.:${PYTHONPATH}
Just specify image file path which you want to upscale. If you get memory error or exit code 137, please try with smaller size image.
Ex. Upscaling input.jpg
python src/inference.py input.jpg
Ex. Upscaling /path/to/input.jpg to /path/to/output.jpg
python src/inference.py /path/to/input.jpg /path/to/output.jpg
SeRanet project studies several network architecture. You can specify which network archtecture to use for SR with -a option.
Ex. use model seranet_v1
python src/inference.py /path/to/input.jpg /path/to/output.jpg -a seranet_v1
GPU makes calculation much faster. Specify -g option is to use GPU.
python src/inference.py /path/to/input.jpg /path/to/output.jpg -g 0
You can construct your own convolutional neural network, and train it!
Put training images[1] inside data/training_images directory. (I used 5000 photo images during the training, see next section "Training images".)
[1]: Currently, images will be cropped to size 232 x 232 during training.
Once prepared training_images, see code/arch/ directory to choose which model to train, and execute below. -g 0 is to use GPU. (For the training, it is highly recommended to use GPU, otherwise training don't finish maybe a month...)
python src/train.py -a model_name -g 0
Training images may be more important than algorithm of deep learning to achieve high quality super resolution. Generally, training images dataset distributed for image classification task are not so high quality in terms of sharpness of the images.
Therefore, I chose PEXELS photos for the training of SeRanet.
The website provides high quality photos under Creative Commons Zero (CC0) license.
Thanks to the pexels team and photographers, I can re-distribute training images dataset which I used. It consists of 5000 medium size PEXELS photos. You can download from below,
- Download PEXELS training_images (jump to google drive, file size around 269MB)
After extract, copy this to data/training_images to start your own training.
The performance of SR for this project is still not matured. You are welcome to suggest any improvement & contribute to this project. If you could get any model which performs better performance, feel free to send me a pull request!