This repo is trying to make a way that can make you image reduce its Demosaicing and Super Resolution executing time by joint them in a model. And also enhance image resolution quality by algorithm.
An usual digital camera will use Color Filter Array which generated by a CMOS, it only can detect one color in each pixel (because if we want detect R,G,B in each pixel, we need to prepare 3 kinds of sensor in each pixel, that will make company cost a lot!), hence we will have a R, G, G, B Bayer input image. Make the 1 dimension input image into 3 dimentions output image is called Demosaicing.
Except demosaic, we also need to consider image resolution. This repo is using a way called Super Resolution to enlarge image and enhance its resolution.
1. Color Filter Array, Bayer Input Data
2. Demosaicing
3. Super Resolution
- BSD200 : 200 Images with 481x321
- Set5 : 5 Images
- Kodak : 24 Images
- BSD100 : 100 Images
- 2021/04/16 在固定各種參數后，以Set5為基準測資
- 使用bilinear的結果:
PSNR:28.73,SSIM:0.753 - 使用bicubic的結果 ：
PSNR:28.58,SSIM:0.744
- 2021/04/16 把patch_size從64改成32，label_size從128改成64
PSNR:28.65,SSIM:0.757
- 2021/04/16 把coarse_output拿掉
PSNR:28.98,SSIM:0.787
- reduce_lr callbacks
- 把reduce_lr加進去:
PSNR:28.72,SSIM:0.724 - 把reduce_lr撤回去:
PSNR:28.87,SSIM:0.738
- 同上設定，batch_size改成16,Model_output只有Residual_Output
PSNR:29.31,SSIM:0.805
- 64，128，30，T91，/255.0，6 residual_block, output=Residual_Output, without reduce_lr
PSNR:29.29,SSIM:0.821
- 把koda也加入training data set中，training_dataset(koda, T91, origin)
- original
PSNR:29.29,SSIM:0.821 - current
PSNR:27.96,SSIM:0.421
-
2021/04/21 做bayer_reverse
-
2021/04/21 計算RGB和BGR的PSNR
-
2021/04/21 testing的圖加上preprocessing (bayer_reverse + resize)
-
加入BSD200
-
2021/04/24 創立一個縮小版本的BSD200，稱為BSD，裡面有隨機從BSD200選出30張的圖片，目的是使用輕量版的dataset測試程式碼的正確性
-
2021/04/24 重要！ image.load_image這個method會使到讀進來的圖片強制變成3通道！ 所以應該要用Image.open來讀取patch image
-
2021/04/24 稍微調整了bayer_reverse里的 R和B通道的寫法，晚點可以調回來
-
2021/04/24 通道對調的寫法很重要 label[:,:,0] = tmp2[:,:,2]， 沒有這個部分的處理的話，藍色和紅色的色域會調換，所以不能
-
2021/04/26 TODO, 使用bicubic
-
2021/04/26 TODO, upsampling interpolation use 'bilinear'
- original
PSNR:28.82,SSIM:0.774 - current
PSNR:28.95,SSIM:0.784
- 2021/04/26 Remove rrp callbacks
- original
PSNR:28.95,SSIM:0.784 - current
PSNR:29.69,SSIM:0.787
- 2021/04/26 Testing kodak_HR dataset
- current
PSNR:29.70,SSIM:0.873
- 2021/04/26 add 2 more Conv layers in the first layer of DCNN part
- original
PSNR:28.75,SSIM:0.787 - current
PSNR:30.25,SSIM:0.813
-
2021/05/03 try to figure out
fusionandensemble. -
2021/05/13 use BSD200 as training dataset, result show that if we add more training dataset will make PSNR improve
- current
PSNR:31.45,SSIM:0.798
- Try to add some layers, and Dense Block into model, dense structure is good at extracting important information from Image/Video.
- Design your own loss function for this purpose. We are doing two great topic (Demosaicing and SR), so create a specific loss function for this model may make output Image quality upgrade.
- Redesign a small model for CFA pixel input data convert to 4 dimensions input image, after that link to DCNN model to continue process. (Implement own model instead of call defined function from library)
- Feel free to contact me if you have any problem or idea!
- Ruofan Zhou, Radhakrishna Achanta, Sabine Süsstrunk, “Deep Residual Network for Joint Demosaicing and Super-Resolution,”IC EPFL, arXiv:1802.06573v1 2018.
- Junkang Zhang, Cheolhong An, Truong Nguyen, “Deep Joint Demosaicing and Super Resolution on High Resolution Bayer Sensor Data,”2018
- W. T. Huang, W. J. Chen, S. C. Tai, “A Sharp Edge-Preserving Joint Color Demosaicking and Zooming Algorithm using Integrated Gradients and An Iterative Back-Projection Technique,”2013.
- DengWen Zhou, WeiMing Dong, Wengang Chen, “Joint Demosaicking and Zooming Using Moderate Spectral Correlation and Consistent Edge Map,”2014.