一种对称残差CNN的图像超分辨率重建方法

doi:10.19665/j.issn1001-2400.2019.05.003

Abstract

Abstract:

The image super-resolution reconstruction methods based on the convolutional neural network have high reconstruction performance, but they have many network parameters and are difficult to train. Also, they are prone to problems such as gradient disappearance and network degradation. To solve these problems, a super-resolution image reconstruction method based on the symmetric residual convolution neural network is proposed. This method integrates symmetry into residual blocks. It realizes local feature fusion by adopting the symmetric connection and extracts as many valuable features as possible. To improve the quality of image reconstruction, the global feature fusion is realized by skip connection. In this method, the peak signal to noise ratio (PSNR) and structural similarity (SSIM) are used as evaluation indexes. The results by the proposed reconstruction method on Set5, Set14, and BSD100 are superior to those by most of other methods in comparison. The average PSNR and SSIM values are improved compared with those methods. Experimental results show that the image reconstructed by this method has clearer textures, richer details and a better subjective visual effect.

Key words: super-resolution reconstruction, convolution neural network, deep learning, symmetric residual network

CLC Number:

TP391

LIU Shudong,WANG Xiaomin,ZHANG Yan. Symmetric residual convolution neural networks for the image super-resolution reconstruction[J].Journal of Xidian University, 2019, 46(5): 15-23.

Figures/Tables 10

数据集	采样因子	Bicubic (PSNR /SSIM)	A+ (PSNR /SSIM)	SRCNN (PSNR /SSIM)	VDSR (PSNR /SSIM)	DRCN (PSNR /SSIM)	LapSRN (PSNR /SSIM)	SymRCN (PSNR /SSIM)
Set5	$×$ 2	33.66/0.929 9	36.54/0.954 4	36.66/0.954 2	37.53/0.958 7	37.63/0.958 8	37.52/0.959 1	37.83/0.9598
	$×$ 3	30.39/0.868 2	32.58/0.908 8	32.75/0.909 0	33.66/0.921 3	33.82/0.922 6	33.81/0.922 0	34.00/0.924 6
	$×$ 4	28.42/0.810 4	30.28/0.860 3	30.48/0.862 8	31.35/0.883 8	31.53/0.885 4	31.54/0.885 2	31.53/0.885 5
Set14	$×$ 2	30.24/0.868 8	32.28/0.905 6	32.42/0.906 3	33.03/0.912 4	33.04/0.911 8	32.99/0.912 4	33.22/0.913 7
	$×$ 3	27.55/0.774 2	29.13/0.818 8	29.28/0.820 9	29.77/0.831 4	29.76/0.831 1	29.79/0.832 5	29.88/0.833 8
	$×$ 4	26.00/0.702 7	27.32/0.749 1	27.49/0.750 3	28.01/0.767 4	28.02/0.767 0	28.09/0.770 0	28.05/0.768 8
BSD100	$×$ 2	29.56/0.843 1	31.21/0.886 3	31.36/0.887 9	31.90/0.896 0	31.85/0.894 2	31.80/0.895 2	32.01/0.897 4
	$×$ 3	27.21/0.738 5	28.29/0.783 5	28.41/0.786 3	28.82/0.797 6	28.80/0.796 3	28.82/0.798 0	28.87/0.799 6
	$×$ 4	25.96/0.667 5	26.82/0.708 7	26.90/0.710 1	27.29/0.725 1	27.23/0.723 3	27.32/0.727 5	27.24/0.724 7
平均值		28.78/0.800 4	30.50/0.841 7	30.64/0.843 1	31.26/0.854 9	31.30/0.854 5	31.30/0.855 8	31.40/0.867 5

数据集	采样因子	A+	SRCNN	VDSR	DRCN	LapSRN	SymRCN
Set5	$×$ 2	0.623	3.389	0.054	0.735	0.032	0.078
	$×$ 3	0.400	3.405	0.062	0.748	0.049	0.040
	$×$ 4	0.281	3.496	0.054	0.735	0.040	0.027
Set14	$×$ 2	1.370	5.134	0.113	1.579	0.035	0.142
	$×$ 3	0.790	5.095	0.122	1.569	0.061	0.068
	$×$ 4	0.630	5.162	0.112	1.526	0.040	0.044
BSD100	$×$ 2	0.921	3.992	0.071	0.983	0.018	0.092
	$×$ 3	0.563	4.140	0.071	0.996	0.037	0.044
	$×$ 4	0.411	3.998	0.071	0.984	0.023	0.028
平均值		0.665	4.201	0.081	1.095	0.037	0.062

References 36

[1]	胡长胜, 詹曙, 吴从中 . 基于深度特征学习的图像超分辨率重建[J]. 自动化学报, 2017,43(5):814-821.
	HU Changsheng, ZHAN Shu, WU Congzhong . Image Super-resolution Based on Deep Learning Features[J]. Acta Automatica Sinica, 2017,43(5):814-821.
[2]	李欣, 韦宏卫, 张洪群 . 结合深度学习的单幅遥感图像超分辨率重建[J]. 中国图象图形学报, 2018,23(2):209-218.
	LI Xin, WEI Hongwei, ZHANG Hongqun . Super-resolution Reconstruction of Single Remote Sensing Image Combined with Deep Learning[J]. Journal of Image and Graphics, 2018,23(2):209-218.
[3]	REN H Y, EI-KHAMY M, LEE J . CT-SRCNN: Cascade Trained and Trimmed Deep Convolutional Neural Networks for Image Super Resolution[C]// Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2018: 1423-1431.
[4]	ZOU W W W, YUEN P C . Very Low Resolution Face Recognition Problem[J]. IEEE Transactions on Image Processing, 2012,21(1):327-340.
[5]	KARRAS T, AILA T, LAINE S , et al. Progressive Growing of GANs for Improved Quality, Stability, and Variation[CA/OL]. [2018-12-20]..
[6]	HARRIS J L . Diffraction and Resolving Power[J]. Journal of the Optical Society of America, 1964,54(7):931-936.
[7]	ZHU S, ZENG B, LIU G , et al. Image Interpolation Based on Non-local Geometric Similarities[C]// Proceedings of the 2015 IEEE International Conference on Multimedia and Expo. Washington: IEEE Computer Society, 2015: 7177417.
[8]	ZHANG K, GAO X, TAO D , et al. Single Image Super-resolution with Non-local Means and Steering Kernel Regression[J]. IEEE Transactions on Image Processing, 2012,21(11):4544-4556.
[9]	DONG C, LOY C C, HE K , et al. Image Super-resolution Using Deep Convolutional Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016,38(2):295-307.
[10]	王世平, 毕笃彦, 刘坤 , 等. 一种多映射卷积神经网络的超分辨率重建算法[J]. 西安电子科技大学学报, 2018,45(4):155-160.
	WANG Shiping, BI Duyan, LIU Kun , et al. Multi-mapping Convolution Neural Network for the Image Super-resolution Algorithm[J]. Journal of Xidian University, 2018,45(4):155-160.
[11]	YANG J C, WRIGHT J, HUANG T S , et al. Image Super-resolution as Sparse Representation of Raw Image Patches[C]// Proceedings of the 2008 26th IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2008: 4587647.
[12]	YANG J C, WRIGHT J, HUANG T S , et al. Image Super-resolution via Sparse Representation[J]. IEEE Transactions on Image Processing, 2010,19(11):2861-2873.
[13]	TIMOFTE R, DE V, GOOL L V . Anchored Neighborhood Regression for Fast Example-based Super-resolution[C]// Proceedings of the 2013 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2013: 1920-1927.
[14]	TIMOFTE R, DE S V, VAN G L . A+: Adjusted Anchored Neighborhood Regression for Fast Super-resolution[C]// Lecture Notes in Computer Science: 9006. Heidelberg: Springer Verlag, 2015: 111-126.
[15]	ZHANG H, PATEL V M . Densely Connected Pyramid Dehazing Network[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 3194-3203.
[16]	ZHANG H, PATEL V M . Density-aware Single Image De-raining Using a Multi-stream Dense Network[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 695-704.
[17]	张金刚, 方圆, 袁豪 , 等. 一种识别表情序列的卷积神经网络[J]. 西安电子科技大学学报, 2018,45(1):150-155.
	ZHANG Jingang, FANG Yuan, YUAN Hao , et al. Multiple Convolutional Neural Networks for Facial Expression Sequence Recognition[J]. Journal of Xidian University, 2018,45(1):150-155.
[18]	许强, 李伟, 占荣辉 , 等. 一种改进的卷积神经网络SAR目标识别算法[J]. 西安电子科技大学学报, 2018,45(5):177-183.
	XU Qiang, LI Wei, ZHAN Ronghui , et al. Improved Algorithm for SAR Target Recognition Based on the Convolutional Neural Network[J]. Journal of Xidian University, 2018,45(5):177-183.
[19]	KIM J, LEE J K, LEE K M . Accurate Image Super-resolution Using Very Deep Convolutional Networks[C]// Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 1646-1654.
[20]	KIM J, LEE J K, LEE K M . Deeply-recursive Convolutional Network for Image Super-resolution[C]// Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 1637-1645.
[21]	LAI W S, HUANG J B, AHUJA N , et al. Deep Laplacian Pyramid Networks for Fast and Accurate Super-resolution[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 5835-5843.
[22]	TAI Y, YANG J, LIU X . Image Super-resolution via Deep Recursive Residual Network[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2790-2798.
[23]	MAO X J, SHEN C H, YANG Y B . Image Restoration Using Very Deep Convolutional Encoder-decoder Networks with Symmetric Skip Connections[C]// Advances in Neural Information Processing Systems 29 - Proceedings of the 2016 Conference. Vancouver: Neural Information Processing Systems Foundation, 2016: 2810-2818.
[24]	HE K, ZHANG X, REN S , et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 770-778.
[25]	陈建廷, 向阳 . 深度神经网络训练中梯度不稳定现象研究综述[J]. 软件学报, 2018,29(7):2071-2091.
	CHEN Jianting, XIANG Yang . Survey of Unstable Gradients in Deep Neural Network Training[J]. Journal of Software, 2018,29(7):2071-2091.
[26]	TAI Y, YANG J, LIU X . Image Super-resolution via Deep Recursive Residual Network[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2790-2798.
[27]	LIM B, SON S, KIM H , et al. Enhanced Deep Residual Networks for Single Image Super-resolution[C]// Proceedings of the 2017 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. Washington: IEEE Computer Society, 2017: 1132-1140.
[28]	AHN N, KANG B, SOHN K A. Fast, Accurate , Lightweight Super-resolution with Cascading Residual Network[C]// Lecture Notes in Computer Science: 11214. Heidelberg: Springer Verlag, 2018: 256-272.
[29]	ZHANG Y, LI K P, LI K , et al. Image Super-resolution Using Very Deep Residual Channel Attention Networks[C]// Lecture Notes in Computer Science: 11211. Heidelberg: Springer Verlag, 2018: 294-310.
[30]	LI J C, FANG F M, MEI K F , et al. Multi-scale Residual Network for Image Super-resolution[C]// Lecture Notes in Computer Science: 11212. Heidelberg: Springer Verlag, 2018: 527-542.
[31]	孙跃文, 李立涛, 丛鹏 , 等. 基于深度学习的辐射图像超分辨率重建方法[J]. 原子能科学技术, 2017,51(5):890-895.
	SUN Yuewen, LI Litao, CONG Peng , et al. Super-resolution Method for Radiation Image Based on Deep Learning[J]. Atomic Energy Science and Technology, 2017,51(5):890-895.
[32]	MARTIN D, FOWLKES C, TAL D , et al. A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics[C]// Proceedings of the 2001 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2001: 416-423.
[33]	JIA Y, SHELHAMER E, DONAHUE J , et al. Caffe: Convolutional Architecture for Fast Feature Embedding[C]// Proceedings of the 2014 ACM Conference on Multimedia. New York: ACM, 2014: 675-678.
[34]	HUANG J B, SINGH A, AHUJA N . Single Image Super-resolution from Transformed Self-exemplars[C]// Proceedings of the 2015 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2015: 5197-5206.
[35]	李浪宇, 苏卓, 石晓红 , 等. 图像超分辨率重建中的细节互补卷积模型[J]. 中国图象图形学报, 2018,23(4):572-582.
	LI Langyu, SU Zhuo, SHI Xiaohong , et al. Mutual-detail Convolution Model for Image Super-resolution Reconstruction[J]. Journal of Image and Graphics, 2018,23(4):572-582.
[36]	HE K, ZHANG X, REN S , et al. Delving Deep into Rectifiers: Surpassing Human-level Performance on Imagenet Classification[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 1026-1034.

Symmetric residual convolution neural networks for the image super-resolution reconstruction

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 36

Related Articles 15

Metrics

Comments

Recommended 10

[1]	LIU Jiawei,ZHANG Wenhui,KOU Xiaoli,LI Yanni. Harnessing adversarial examples via input denoising and hidden information restoring [J]. Journal of Xidian University, 2021, 48(6): 23-31.
[2]	DONG Ruchan,JIAO Licheng,ZHAO Jin,SHEN Weiyan. Application of the deep fusion mechanism in object detection of remote sensing images [J]. Journal of Xidian University, 2021, 48(5): 128-138.
[3]	LI Peng,FENG Cunqian,XU Xuguang,TANG Zixiang. Ballistic target fretting classification network based on Bayesian optimization [J]. Journal of Xidian University, 2021, 48(5): 139-148.
[4]	YAN Jia,CAO Yudong,REN Jiaxing,CHEN Donghao,LI Xiaohui. Deep asymmetric compression Hashing algorithm [J]. Journal of Xidian University, 2021, 48(5): 212-221.
[5]	NING Yang,DU Jianchao,HAN Shuo,YANG Chuankai. Fire segmentation based on the improved DeeplabV3+ and the analytical method for fire development [J]. Journal of Xidian University, 2021, 48(5): 38-46.
[6]	ZHOU Peng,YANG Jun. Semantic segmentation of remote sensing images based on neural architecture search [J]. Journal of Xidian University, 2021, 48(5): 47-57.
[7]	QI Yanjun,KONG Yueping,WANG Jiajing,ZHU Xudong. Gait recognition method combining LSTM and CNN [J]. Journal of Xidian University, 2021, 48(5): 78-85.
[8]	SONG Jianfeng,MIAO Qiguang,WANG Chongxiao,XU Hao,YANG Jin. Multi-scale single object tracking based on the attention mechanism [J]. Journal of Xidian University, 2021, 48(5): 110-116.
[9]	ZHANG Yuhao,CHENG Peitao,ZHANG Shuhao,WANG Xiumei. Lightweight image super-resolution with the adaptive weight learning network [J]. Journal of Xidian University, 2021, 48(5): 15-22.
[10]	HUI Haisheng,ZHANG Xueying,WU Zelin,LI Fenglian. Method for stroke lesion segmentation using the primary-auxiliary path attention compensation network [J]. Journal of Xidian University, 2021, 48(4): 200-208.
[11]	SUN Haojie,LI Miaoyu,ZHANG Panpan,XU Pengfei. Self-supervised facial asymmetry learning for automatic evaluation of facial paralysis [J]. Journal of Xidian University, 2021, 48(3): 115-122.
[12]	ZHANG Hua,GAO Haoran,YANG Xingguo,LI Wenmin,GAO Fei,WEN Qiaoyan. TargetedFool:an algorithm for achieving targeted attacks [J]. Journal of Xidian University, 2021, 48(1): 149-159.
[13]	YANG Hongyu,ZENG Renyun. Method for assessment of network security situation with deep learning [J]. Journal of Xidian University, 2021, 48(1): 183-190.
[14]	ZHANG Lu,SUN Rong,LIU Jingwei. Cloned piggybacking framework for distributed storage [J]. Journal of Xidian University, 2020, 47(6): 139-147.
[15]	HU Jianwei,ZHAO Wei,CUI Yanpeng,CUI Junjie. PHP code vulnerability mining technology based on theimproved ASTNN [J]. Journal of Xidian University, 2020, 47(6): 164-173.