Deep Cyclic Generative Adversarial Residual Convolutional Networks for Real Image Super-Resolution (SRResCycGAN)

An official PyTorch implementation of the SRResCycGAN network as described in the paper Deep Cyclic Generative Adversarial Residual Convolutional Networks for Real Image Super-Resolution. This work is participated in the AIM 2020 Real-Image Super-resolution challenge track-3 at the high x4 upscaling factor.

Abstract

Recent deep learning based single image super-resolution (SISR) methods mostly train their models in a clean data domain where the low-resolution (LR) and the high-resolution (HR) images come from noise-free settings (same domain) due to the bicubic down-sampling assumption. However, such degradation process is not available in real-world settings. We consider a deep cyclic network structure to maintain the domain consistency between the LR and HR data distributions, which is inspired by the recent success of CycleGAN in the image-to-image translation applications. We propose the Super-Resolution Residual Cyclic Generative Adversarial Network (SRResCycGAN) by training with a generative adversarial network (GAN) framework for the LR to HR domain translation in an end-to-end manner. We demonstrate our proposed approach in the quantitative and qualitative experiments that generalize well to the real image super-resolution and it is easy to deploy for the mobile/embedded devices. In addition, our SR results on the AIM 2020 Real Image SR Challenge datasets demonstrate that the proposed SR approach achieves comparable results as the other state-of-art methods.

Spotlight Video

Pre-trained Models

Datasets	SRResCycGAN
NTIRE2020 RWSR	Sensor noise (σ = 8)
NTIRE2020 RWSR	JPEG compression (quality=30)
NTIRE2020 RWSR	Unknown corruptions
AIM2020 RISR	Real image corruptions

BibTeX

@InProceedings{Umer_2020_ECCVW,
               author = {Muhammad Umer, Rao and Micheloni, Christian},
               title = {Deep Cyclic Generative Adversarial Residual Convolutional Networks for Real Image Super-Resolution},
               booktitle = {Proceedings of the European Conference on Computer Vision (ECCV) Workshops},
               month = {August},
               year = {2020}
              }

Quick Test

This model can be run on arbitrary images with a Docker image hosted on Replicate: https://beta.replicate.ai/RaoUmer/SRResCycGAN. Below are instructions for how to run the model without Docker:

Dependencies

• Python 3.7 (version >= 3.0)
• PyTorch >= 1.0 (CUDA version >= 8.0 if installing with CUDA.)
• Python packages: pip install numpy opencv-python

Train models

• The SR training code is based on the SRResCGAN.

Test models

1. Clone this github repository as the following commands:

git clone https://github.com/RaoUmer/SRResCycGAN
cd SRResCycGAN
cd srrescycgan_code_demo

2. Place your own low-resolution images in the ./srrescycgan_code_demo/LR folder. (There are two sample images i.e. LR_006 and LR_014).
3. Download the pretrained models from Pre-trained Models section. Place the models in ./srrescycgan_code_demo/trained_nets_x4.
4. Run the test. You can config in the test_srrescycgan.py.

python test_srrescgan.py

5. The results are in the ./srrescycgan_code_demo/sr_results_x4 folder.

SRResCycGAN Architecture

Overall Representative diagram

Quantitative Results

The x4 SR quantitative results comparison of our method with others over the DIV2K validation-set (100 images). The best performance is shown in red and the second best performance is shown in blue.

The AIM2020 Real Image SR Challenge Results (x4)

Team	PSNR↑	SSIM↑	Weighed_score↑
Baidu	31.3960	0.8751	0.7099 (1)
ALONG	31.2369	0.8742	0.7076 (2)
CETC-CSKT	31.1226	0.8744	0.7066 (3)
SR-IM	31.2369	0.8728	0.7057
DeepBlueAI	30.9638	0.8737	0.7044
JNSR	30.9988	0.8722	0.7035
OPPO_CAMERA	30.8603	0.8736	0.7033
Kailos	30.8659	0.8734	0.7031
SR_DL	30.6045	0.8660	0.6944
Noah_TerminalVision	30.5870	0.8662	0.6944
Webbzhou	30.4174	0.8673	0.6936
TeamInception	30.3465	0.8681	0.6935
IyI	30.3191	0.8655	0.6911
MCML-Yonsei	30.4201	0.8637	0.6906
MoonCloud	30.2827	0.8644	0.6898
qwq	29.5878	0.8547	0.6748
SrDance	29.5952	0.8523	0.6729
MLP_SR (ours)	28.6185	0.8314	0.6457
EDSR	28.2120	0.8240	0.6356
RRDN_IITKGP	27.9708	0.8085	0.6201
congxiaofeng	26.3915	0.8258	0.6187

Visual Results

DIV2K Validation-set (100 images)

Here are the SR resutls comparison of our method on the DIV2K validation-set images.

Real-Image SR Challenge dataset images (Track-3)

Validation-set

You can download all the SR resutls of our method on the AIM 2020 Real-Image SR validation-set from the Google Drive: SRResCycGAN.

Test-set

You can download all the SR resutls of our method on the AIM 2020 Real-Image SR test-set from the Google Drive: SRResCycGAN.