基于压缩卷积神经网络的图像超分辨率算法

doi:10.16180/j.cnki.issn1007-7820.2020.05.001

摘要/Abstract

摘要：

为了有效提高深度图像的分辨率,文中借鉴经典SqueezeNet网络结构,提出一种基于Fire Module的卷积神经网络模型。该算法实现了直接从低分辨率图像到高分辨率图像的映射和转化,其中Fire Module作为网络的非线性映射模块,在减少参数的同时可学习图像的深层特征。为了避免插值预处理,在网络的输出层引入反卷积层,实现3倍上采样和高分辨率图像的输出。实验表明,采用该基于Fire Module的卷积神经网络模型的反卷积算法得到的超分辨率图像细节更加丰富,客观指标PSNR值和SSIM值的评价也明显优于其他算法。

关键词: 图像处理, 超分辨重建, 卷积神经网络, 反卷积, 残差块, 层次块

Abstract:

In order to effectively improve the resolution of depth image, the study proposed a convolutional neural network model based on the Fire Module by referring to the classic SqueezeNet network structure. The proposed algorithm implemented mapping and transformation directly from low-resolution images to high-resolution images. As a nonlinear mapping module of the network, Fire Module learned the deep features of the image while reducing the parameters. To avoid interpolation preprocessing, a deconvolution layer was introduced in the output layer of the network to achieve a final 3 times up-sampling and high resolution image output. Experiments showed that the super-resolution image obtained by the deconvolution algorithm of the convolutional neural network model based on Fire Module was richer in detail, and the evaluation of objective index PSNR value and SSIM value was also superior to other algorithms.

Key words: image processing, super-resolution reconstruction, convolution neural network, deconvolution, residual block, inception block

中图分类号:

TP391

秦兴,高晓琪,陈滨. 基于压缩卷积神经网络的图像超分辨率算法[J]. 电子科技, 2020, 33(5): 1-8.

QIN Xing,GAO Xiaoqi,CHEN Bin. Image Super-resolution Algorithm Based on SqueezeNet Convolution Neural Network[J]. Electronic Science and Technology, 2020, 33(5): 1-8.

图/表 13

图1

图2

图3

表1

图4

图5

表2

图6

图7

表3

表4

图8

图9

参考文献 18

[1]	曾凯, 丁世飞 . 图像超分辨率重建的研究进展[J]. 计算机工程与应用, 2017,53(16):29-35.
	Zeng Kai, Ding Shifei . Advances in image super-resolution reconstruction[J]. Computer Engineering and Applica-tions, 2017,53(16):29-35.
[2]	沈丽, 韩彦芳 . 基于稀疏表示的超分辨率图像重建[J]. 电子科技, 2015,28(9):144-148
	Shen Li, Han Yanfang . Image super-resolution reconstruction based on sparse representatio[J]. Electronic Science and Technology, 2015,28(9):144-148.
[3]	Irani M, Peleg S . Super resolution from image sequences[J]. ICPR-C, 1990,90(11):115-120.
[4]	李影, 徐伯庆 . 一种基于压缩感知的迭代重建算法[J]. 电子科技, 2016,29(11):129-134.
	Li Ying, Xu Boqing . An iterative image reconstruction algorithm based on compressed sensing[J]. Electronic Science and Technology, 2016,29(11):129-134.
[5]	Li Y M, Pedrycz W . Fuzzy finite qutomata and fuzzy regular expressions with membership values in lattice ordered monoids[J]. Fuzzy Sets and Systems, 2005,156(10):68-92.
[6]	张磊, 杨建峰, 薛彬 , 等. 改进的最大后验概率估计法实现单幅图像超分辨率重建[J]. 激光与光电子学进展, 2011,48(1):78-85.
	Zhang Lei, Yang Jianfeng, Xue Bin , et al. Modified MAP algorithm for single frame super-resolution reconstruction[J]. Laser & Optoelectronics Progress, 2011,48(1):78-85.
[7]	Chang H, Yeung D Y, Xiong Y. Super-resolution through neighbor embedding [C]. Washington DC:Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004.
[8]	Gao X, Zhang K, Tao D , et al. Image super-resolution with sparse neighbor embedding[J]. IEEE Transactions on Image Processing, 2012,21(7):78-85.
[9]	Timofte R, De V, Gool L V. Anchored neighborhood regression for fast example-based super-resolution [C]. Sydney:IEEE International Conference on Computer Vision, 2013.
[10]	Yang J, Wright J, Huang T S et al.Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010,19(7):2861-2873.
[11]	Dong C, Loy C C, He K , et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Anal Mach Intell, 2014,38(2):295-307.
[12]	Dong C, Deng Y, Chen C L, et al. Compression artifacts reduction by a deep convolutional network [C]. Santiago:IEEE International Conference on Computer Vision, 2015.
[13]	王一宁, 秦品乐, 李传朋 , 等. 基于残差神经网络的图像超分辨率改进算法[J]. 计算机应用, 2018,22(1):246-254.
	Wang Yining, Qin Pinle, Li Chuanpeng . Improved algorithm of image super resolution based on residual neural network[J]. Journal of Computer Applications, 2018,22(1):246-254.
[14]	Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks [C]. Las Vegas:Computer Vision and Pattern Recognition, 2016.
[15]	Iandola F N, Han S. SqueezeNet:AlexNet-level accuracy with 50× fewer parameters and <0.5MB model size [C]. San Juan:International Conference on Learning Representations, 2016.
[16]	He Kaiming, Sun Jian. Convolutional neural networks at constrained time cost [C]. Boston:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[17]	Timofte R, De V, Gool L V. Anchored neighborhood regression for fast example-based super-resolution [C]. Sydney:Processing of the IEEE International Conference on Computer Vision, 2013.
[18]	周壮, 鲍伟龙 . 超分辨率成像系统的FPGA控制研究[J]. 电子科技, 2011,24(10):122-126.
	Zhou Zhuang, Bao Weilong . Research on FPGA control in super-resolution color imaging system[J]. Electronic Science and Technology, 2011,24(10):122-126.

Name	Number	Size	Pad
Conv₁	56	5×5	0
Conv₂	12	1×1	0
Conv_{2_1}	12	1×1	0
Conv_{2_2}	12	3×3	1
Conv₃	56	1×1	0
Conv₄	1	9×9	4

	PSNR			SSIM
	Art	Books	Dolls	Art	Books	Dolls
bicubic	31.14	31.21	32.59	0.928 0	0.958 5	0.952 3
Line3-1	33.15	32.06	33.41	0.956 9	0.965 9	0.958 6
Line3	33.20	32.30	33.53	0.957 2	0.966 6	0.959 1
FireNet1-3-5	33.28	32.32	33.61	0.959 1	0.968 0	0.960 3
FireNet1-3	33.22	32.20	33.48	0.956 9	0.966 2	0.958 3

Image	Index	Bicubic	ScSR	ANR	SRCNN	Proposed
Butterfly	SSIM	0.821 4	0.857 4	0.873 6	0.902 9	0.920 0
Butterfly	PSNR	24.03	25.46	25.98	27.57	28.49
Woman	SSIM	0.889 2	0.909 4	0.917 1	0.923 9	0.930 0
Woman	PSNR	28.56	29.86	30.45	30.91	31.61
Comic	SSIM	0.688 9	0.756 9	0.760 3	0.776 1	0.786 9
Comic	PSNR	23.11	23.89	24.00	24.31	24.54
Face	SSIM	0.797 7	0.819 8	0.823 1	0.820 1	0.827 6
Face	PSNR	32.80	33.38	33.57	33.52	33.72
zebra	SSIM	0.795 0	0.841 2	0.845 6	0.794 6	0.852 2
zebra	PSNR	26.17	26.62	28.54	28.82	29.09

Image	Index	Bicubic	ScSR	ANR	SRCNN	Proposed
Art	SSIM	0.871 9	0.891 5	0.902 1	0.907 5	0.915 5
Art	PSNR	30.61	31.66	32.06	33.64	33.14
Books	SSIM	0.807 7	0.818 5	0.835 3	0.841 5	0.847 2
Books	PSNR	28.46	29.21	29.51	29.96	30.17
Dolls	SSIM	0.853 3	0.872 5	0.885 8	0.889 3	0.896 8
Dolls	PSNR	30.04	31.08	31.47	31.85	32.19
Laundry	SSIM	0.839 1	0857 7	0.870 8	0.877 2	0.885 9
Laundry	PSNR	28.62	29.80	30.16	30.89	31.49
Reindeer	SSIM	0.879 0	0.884 7	0.899 4	0.905 4	0.910 9
Reindeer	PSNR	31.51	32.66	32.85	33.62	33.95