条件约束下的自我注意生成对抗网络

doi:10.19665/j.issn1001-2400.2019.06.023

摘要/Abstract

摘要：

为了解决生成对抗网络中因生成图像的特征信息表示不足而导致生成效果特征不明显、图像的关键特征信息模糊的问题, 提出了一种条件自我注意生成对抗网络的图像生成方法。该网络结合自我注意生成对抗网络的优点,向生成器和判别器中添加附加条件特征,明确指示模型生成对应的标志性类别信息,将数据的具体维度与语义特征关联起来,用这种方法提取其中的生成模型,使生成特定类型的图像的特征表示更加贴合原始数据分布。实验结果表明,所提出的方法在CelebA和MNIST数据集上的弗雷歇距离值相比较于自我注意生成对抗网络分别约提高了1.26和2.47。验证了所提出的方法相比较于其他的监督类生成模型可以有效地提升图像的质量效果以及多样性,并且可以有效地加快网络的收敛速度。

关键词: 生成对抗网络, 条件特征, 自注意力, 图像生成

Abstract:

In order to solve the problem that the feature information of the generated image against the network is insufficient, so that the generated effect characteristic is not obvious, and the key feature information of the image is blurred, this paper proposes an image generating method for a conditional self-attention generative adversarial network. The network combines the advantages of the self-attention generative adversarial network, and adds additional conditional features to the generator and the discriminator. It is explicitly indicated that the model can generate corresponding iconic category information. The specific dimensions of the data are related to the semantic features. In this way, the generation model is extracted, so that the feature representations of the images of a particular type are more closely matched to the original data distribution. Experimental results show that the FID values of the proposed method on the CelebA and MNIST data sets are increased by 1.26 and 2.47, respectively, compared with the self-attention generative confrontation network. It is verified that compared with other supervised class generation models, the proposed method can effectively improve the image quality and diversity, and can make the network converge faster.

Key words: generative adversarial network, condition feature, self-attention, image generation

中图分类号:

TP391

贾宇峰,马力. 条件约束下的自我注意生成对抗网络[J]. 西安电子科技大学学报, 2019, 46(6): 163-171.

JIA Yufeng,MA Li. Self-attention generative adversarial network with the conditional constraint[J]. Journal of Xidian University, 2019, 46(6): 163-171.

图/表 10

图1

图2

图3

图4

图5

图6

图7

图8

图9

表1

参考文献 28

[1]	RANZATO M, KRIZHEVSKY A, HINTON G E , et al. Factored 3-way Restricted Boltzmann Machines for Modeling Natural Images[J]. Journal of Machine Learning Research, 2010,9:621-628.
[2]	VINCENT P, LAROCHELLE H, BENGIO Y , et al. Extracting and Composing Robust Features with Denoising Autoencoders[C]//Proceedings of the 2008 25th International Conference on Machine Learning. New York: ACM, 2008: 1096-1103.
[3]	KINGMA D P, WELLING M . Auto-Encoding Variational Bayes[C]//Proceedings of the 2014 2nd International Conference on Learning Representations. San Diego: ICLR, 2014: 149797.
[4]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M , et al. Generative Adversarial Nets[C]//Advances in Neural Information Processing Systems. Vancouver: Neural Information Processing Systems Foundation, 2014: 2672-2680.
[5]	RATLIFF L J, BURDEN S A, SASTRY S S , et al. Characterization and Computation of Local Nash Equilibria in Continuous Games[C]//Proceedings of the 2013 51st Annual Allerton Conference on Communication, Control, and Computing. Washington: IEEE Computer Society, 2013: 917-924.
[6]	BROCK A, DONAHUE J, SIMONYAN K , et al. Large Scale GAN Training for High Fidelity Natural Image Synconfproc[C]//Proceedings of the 2019 7th International Conference on Learning Representation. San Diego: ICLR, 2019: 149936.
[7]	WANG Q, FAN H J, SUN G , et al. Laplacian Pyramid Adversarial Network for Face Completion[J]. Pattern Recognition, 2019,88:493-505. doi: 10.1016/j.patcog.2018.11.020
[8]	LEE H Y, TSENG H Y, HUANG J B , et al. Diverse Image-to-image Translation via Disentangled Representations[C]//Lecture Notes in Computer Science: 11205. Heidelberg: Springer Verlag, 2018: 36-52.
[9]	ZHU J Y, PARK T, ISOLA P , et al. Unpaired Image-to-image Translation using Cycle-consistent Adversarial Networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2242-2251.
[10]	LIU Z, CHAI X, CHEN X , et al. Deep Memory and Prediction Neural Network for Video Prediction[J]. Neurocomputing, 2019,331:235-241. doi: 10.1371/journal.pone.0225317 pmid: 31725778
[11]	CHEN X Y, XU C, YANG X K , et al. Gated-GAN: Adversarial Gated Networks for Multi-collection Style Transfer[J]. IEEE Transactions on Image Processing, 2019,28(2):546-560. doi: 10.1109/TIP.2018.2869695 pmid: 30222565
[12]	LI Y H, ZHANG T Y, HAN X , et al. Image Style Transfer in Deep Learning Networks[C]//Proceedings of the 2018 5th International Conference on Systems and Informatics. Piscataway: IEEE, 2019: 8599501.
[13]	VU T, LUU T M, YOO C D , et al. Perception-enhanced Image Super-resolution via Relativistic Generative Adversarial Networks[C]//Lecture Notes in Computer Science: 11133. Heidelberg: Springer Verlag, 2019: 98-113.
[14]	ZHANG H, XU T, LI H S , et al. StackGAN: Text to Photo-realistic Image Synconfproc with Stacked Generative Adversarial Networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 5908-5916.
[15]	ZHANG H, XU T, LI H S , et al. StackGAN++: Realistic Image Synjournal with Stacked Generative Adversarial Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019,41(8):1947-1962. doi: 10.1109/TPAMI.2018.2856256 pmid: 30010548
[16]	MEHRALIAN M, KARASFI B . RDCGAN: Unsupervised Representation Learning with Regularized Deep Convolutional Generative Adversarial Networks[C]//Proceedings of the 2018 9th Conference on Artificial Intelligence and Robotics and 2nd Asia-Pacific International Symposium. Piscataway: IEEE, 2018: 31-38.
[17]	QUAN T M, NGUYEN-DUC T, JEONG W K . Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network with a Cyclic Loss[J]. IEEE Transactions on Medical Imaging, 2018,37(6):1488-1497. doi: 10.1109/TMI.2018.2820120 pmid: 29870376
[18]	ARJOVSKY M, CHINTALA S, BOTTOU L . Wasserstein GAN[J/OL]. [2019-05-20]. https://arxiv.org/abs/1701.07875.
[19]	GULRAJANI I, AHMED F, ARJOVSKY M , et al. Improved Training of Wasserstein GAN[C]//Advances in Neural Information Processing Systems. Vancouver: Neural Information Processing System Foundation, 2017: 5768-5778.
[20]	MAO X, LI Q, XIE H , et al. Least Squares Generative Adversarial Networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2813-2821.
[21]	ZHAO J B, MATHIEU M, LECUN Y . Energy-based Generative Adversarial Network[C]//Proceedings of the 2017 5th International Conference on Learning Representations. San Diego: ICLR, 2017: 149804.
[22]	BERTHELOT D, SCHUMM T, METZ L . BEGAN: Boundary Equilibrium Generative Adversarial Networks[J/OL]. [2019-05-20]. https://arxiv.org/abs/1703.10717
[23]	LI Y, XIAO N, OUYANG W . Improved Boundary Equilibrium Generative Adversarial Networks[J]. IEEE Access, 2018,6:11342-11348. doi: 10.1109/ACCESS.2018.2804278
[24]	ZHANG H, GOODFELLOW I, METAXAS D , et al. Self-Attention Generative Adversarial Networks[J/OL]. [2019-05-20]. https://arxiv.org/abs/1805.08318.
[25]	MIRZA M, OSINDERO S . Conditional Generative Adversarial Nets[J]. [J/OL]. [2019-05-20]. https://arxiv.org/abs/1411.1784.
[26]	LIU Z, LUO P, WANG X , et al. Deep Learning Face Attributes in the Wild[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 3730-3738.
[27]	LECUN Y, CORTES C, BURGES C J C , The MNIST Database of Handwritten Digits[DB/OL].[2019-05-12]. http://yann.lecun.com/exdb/mnist/.
[28]	陈晓范, 申海杰, 边倩 , 等. 结合注意力机制的人脸超分辨率重建[J]. 西安电子科技大学学报, 2019,44(3):148-153.
	CHEN Xiaofan, SHEN Haijie, BIAN Qian , et al. Face Image Super-resolution with an Attention Mechanism[J]. Journal of Xidian University, 2019,44(3):148-153.

数据集	CGAN	DCGAN	WGAN	SAGAN	C-SAGAN
CelebA	21.82	18.69	16.47	10.51	9.25
MNIST	19.05	14.74	10.28	5.84	3.37