生成对抗网络用于视频去模糊

doi:10.19665/j.issn1001-2400.2019.06.016

摘要/Abstract

摘要：

针对因拍摄设备抖动或目标运动而产生的视频模糊问题,提出了一种基于生成对抗网络和马尔可夫判别网络的新的视频去模糊方法。文中将基于像素空间的损失函数与基于特征空间的损失函数相结合,并依据马尔可夫判别网络设计了一种新的判别网络,促进了网络对图像纹理细节的学习,使得生成的清晰图像质量得到了显著提升。将文中方法与同类方法分别在测试集和真实数据上进行了定性定量的比较,实验结果表明,经文中方法去模糊后的图像具有更高的峰值信噪比和更丰富的细节信息。

关键词: 视频去模糊, 生成对抗网络, 马尔可夫判别网络

Abstract:

A video deblurring network based on the generative adversarial network and the Markovian discriminator is proposed to solve the video deblurring problem, which is caused by camera shaking or object movement. In this paper, we combine the pixel-space and feature-space loss, and design a discriminator based on the Markovian discriminator, which promotes the learning of image texture details and improves the quality of the generated image. The proposed method and the state-of-the-art methods are compared qualitatively and quantitatively on the test set and real video set, respectively. Experimental results indicate that the image deblurred by the proposed method has a higher peak signal-to-noise ratio and richer details.

Key words: video deblurring, generative adversarial network, Markovian discriminator

中图分类号:

TP391

申海杰,边倩,陈晓范,王振铎,田新志. 生成对抗网络用于视频去模糊[J]. 西安电子科技大学学报, 2019, 46(6): 112-117.

SHEN Haijie,BIAN Qian,CHEN Xiaofan,WANG Zhenduo,TIAN Xinzhi. Video deblurring using the generative adversarial network[J]. Journal of Xidian University, 2019, 46(6): 112-117.

图/表 5

图1

表1

图2

图3

表2

参考文献 16

[1]	SU S, DELBRACIO M, WANG J , et al. Deep Video Deblurring for Hand-held Cameras[C]//Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 237-246.
[2]	CHO S, WAND J, LEE S , et al. Video Deblurring for Hand-held Cameras Using Patch-based Synjournal[J]. ACM Transactions on Graphics, 2012,31(4):64.
[3]	DEBRACIO M, SAPIRO G . Hand-held Video Deblurring via Efficient Fourier Aggregation[J]. IEEE Transactions on Computational Imaging, 2015,1(4):270-283. doi: 10.1109/TCI.2015.2501245
[4]	TAO X, GAO H, SHEN X , et al. Scale-recurrent Network for Deep Image Deblurring[C]//Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 8174-8182.
[5]	KUPYN O, BUDZAN V, MYKHAILCH M , et al. Deblurgan: Blind Motion Deblurring Using Conditional Adversarial Networks[C]//Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 8183-8192.
[6]	WU M, LI W, GONG W . Two-frame Convolutional Neural Network for Blind Motion Image Deblurring[J]. Journal of Computer-Aided Design and Computer Graphics, 2018,30(12):2327-2334. doi: 10.3724/SP.J.1089.2018.17163
[7]	ZHANG K, LUO W, ZHONG Y , et al. Adversarial Spatio-temporal Learning for Video Deblurring[J]. IEEE Transactions on Image Processing, 2019,28(1):291-301. doi: 10.1109/TIP.2018.2867733 pmid: 30176588
[8]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M , et al. Generative Adversarial Nets[C]//Advances in Neural Information Processing Systems 27: 3. Vancouver: Neural Information Processing Systems Foundation, 2014: 2672-2680.
[9]	ISOLA P, ZHU J Y, ZHOU T , et al. Image-to-image Translation with Conditional Adversarial Networks[C]//Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 5967-5976.
[10]	LI C, WAND M . Precomputed Real-time Texture Synconfproc with Markovian Generative Adversarial Networks[C]//Lecture Notes in Computer Science: 9907. Heidelberg: Springer Verlag, 2016: 702-716.
[11]	ARJOVSKY M, CHINTALA S, BOTTOU L . Wasserstein GAN[J/OL]. [2019-5-16]. https://arxiv.org/abs/1701.07875.
[12]	PATHAK D, KRAHENBUHL P, DONAHUE J , et al. Context Encoders: Feature Learning by Inpainting[C]//Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 2536-2544.
[13]	JOHNSON J, ALAHI A, FEI-FEI L . Perceptual Losses for Real-time Style Transfer and Super-resolution[C]//Lecture Notes in Computer Science: 9906. Heidelberg: Springer Verlag, 2016: 694-711.
[14]	TIELEMAN T, HINTON G . Lecture 6.5-rmsprop: Divide the Gradient by a Running Average of Its Recent Magnitude[J]. Coursera: Neural Networks for Machine Learning, 2012,4:26-30.
[15]	ZHANG R, ISOLA P, EFROS A A , et al. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric[C]//Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 586-595.
[16]	MOORTHY A K, BOVIK A C . A Two-step Framework for Constructing Blind Image Quality Indices[J]. IEEE Signal Processing Letters, 2010,17(5):513-516. doi: 10.1109/LSP.2010.2043888

方法	文献[4]	文献[5]	文献[1]	文中方法
R_PSNR/dB	30.98	30.22	32.25	33.62
R_SSIM	0.928 5	0.897 6	0.942 3	0.957 5
感知距离^[15]	0.056 8	0.062 5	0.045 2	0.032 8