一种基于GAN的轻量级水墨画风格迁移模型

doi:10.16180/j.cnki.issn1007-7820.2023.02.012

Abstract

Abstract:

Current style transfer methods are mainly suitable for photos or Western paintings. Due to the inherent differences between Chinese and Western paintings, existing algorithms cannot generate satisfactory results when applied to the style transfer task of Chinese ink paintings. Consequently, a novel style transfer method based on GAN for ink painting is proposed in the study. This method combines AdaIN, style-attention module and perceptual loss to learn the style features of ink painting more accurately, thereby solving the problem that the generated results of ink painting have poor quality. According to the qualitative analysis and quantitative evaluation, experimental results show that this method has better performance and the generated results have higher visual quality. Compared with the baseline, the method reduces the number of parameters by about 55% and the training time by about 60%.

Key words: ink painting, style transfer, generative adversarial networks, attention mechanism, ChipGAN, lightweight, AdaIN, perceptual loss

CLC Number:

TP391

ZHAO Jin,LI Feifei. A GAN-Based Lightweight Style Transfer Model for Ink Painting[J].Electronic Science and Technology, 2023, 36(2): 81-86.

Figures/Tables 5

References 19

[1]	Sheng J, Li Y. Classification of traditional Chinese paintings using a modified embedding algorithm[J]. Journal of Electronic Imaging, 2019, 28(2):1-12.
[2]	Sheng J, Li Y. Style-based classification of Chinese ink and wash paintings[J]. Optical Engineering, 2013, 52(9): 1-9.
[3]	LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444. doi: 10.1038/nature14539
[4]	Gatys L A, Ecker A S, Bethge M. Image style transfer using convolutional neural networks[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[5]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]. San Diego: International Conference on Learning Representations, 2015.
[6]	Huang X, Belongie S. Arbitrary style transfer in real-time with adaptive instance normalization[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision, 2017.
[7]	Goodfellow I, Pouget Abadie J, Mirza M, et al. Generative adversarial nets[C]. Montreal: Annual Conference on Neural Information Processing Systems, 2014.
[8]	Isola P, Zhu J Y, Zhou T H, et al. Image-to-image translation with conditional adversarial networks[C]. Hawaii: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[9]	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. Venice: Proceedings of the IEEE Conference on International Conference on Computer Vision, 2017.
[10]	He B, Gao F, Ma D, et al. ChipGAN: A generative adversarial network for chinese ink wash painting style transfer[C]. Seoul: Proceedings of the Twenty-sixth ACM International Conference on Multimedia, 2018.
[11]	Lee H Y, Tseng H Y, Huang J B, et al. Diverse image-to-image translation via disentangled representations[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018.
[12]	Wang X, Yu K, Wu S, et al. ESRGAN: Enhanced super-resolution generative adversarial networks[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018.
[13]	向晴, 袁健华. 基于多层次判别器的图像翻译模型[J]. 软件, 2020, 41(3):11-17.
	Xiang Qing, Yuan Jianhua. Image-to-image translation based on multi-layer discriminator[J]. Software, 2020, 41(3):11-17.
[14]	佟博, 刘韬, 刘畅. 基于生成对抗网络的轴承失效信号生成研究[J]. 电子科技, 2020, 33(4):28-34.
	Tong Bo, Liu Tao, Liu Chang. Research on bearing failure signal generation based on generative adversarial networks[J]. Electronic Science and Technology, 2020, 33(4):28-34.
[15]	Wang X, Girshick R, Gupta A, et al. Non-local neural networks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[16]	Mao X D, Li Q, Xie H R, et al. Least squares generative adversarial networks[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision, 2017.
[17]	Johnson J, Alahi A, Li F F. Perceptual losses for real time style transfer and super-resolution[C]. Amsterdam: European Conference on Computer Vision, 2016.
[18]	Heusel M, Ramsauer H, Unterthiner T, et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium[C]. Long Beach: Annual Conference on Neural Information Processing Systems, 2017.
[19]	Bińkowski M, Sutherland D J, Arbel M, et al. Demystifying MMD GANs[C]. Vancouver: International Conference on Learning Representations, 2018.

	FID	KID
ChipGAN	203.30	7.61
本文模型	201.00	6.80

	参数量/MB	训练时间/h
ChipGAN	11.378	55.2
本文模型	5.100	21.7

A GAN-Based Lightweight Style Transfer Model for Ink Painting

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 19

Related Articles 15

Metrics

Comments

Recommended 10

[1]	ZUO Bin,LI Feifei. An Effective Segmentation Method for COVID-19 CT Image Based on Attention Mechanism and Inf-Net [J]. Electronic Science and Technology, 2023, 36(2): 22-28.
[2]	SUN Kang,XUAN Xuyang,LIU Penghui,ZHAO Laijun,LONG Jie. Partial Discharge Pattern Recognition of Cable Based on CNN-DCGAN under Small Data [J]. Electronic Science and Technology, 2022, 35(7): 7-13.
[3]	CHEN Jinhong,CHEN Wei,YIN Zhong. Semantic Segmentation of Streetscape Based on Improved ExfuseNet [J]. Electronic Science and Technology, 2022, 35(6): 28-34.
[4]	DAI Jiangtao,GAO Bo,WAN Jiajun. Design of DDS Communication Middleware for Resource Limited Devices [J]. Electronic Science and Technology, 2022, 35(5): 47-55.
[5]	Chaowei LIN,Feifei LI,Qiu CHEN. Globaland Local Scene Representation Method Based on Deep Convolutional Features [J]. Electronic Science and Technology, 2022, 35(4): 20-27.
[6]	Lingyu JI,Yongbin GAO,Chenglu ZHAO,Xianhua TANG,Kaicheng XU,Jiacheng XU. CTA Segmentation Algorithm of Abdominal Artery Based on 3D Fully Convolutional Network [J]. Electronic Science and Technology, 2022, 35(3): 38-44.
[7]	CHU Ping,NI Wei. Design of FPGA-Based SqueezeNet Inference Accelerator [J]. Electronic Science and Technology, 2022, 35(2): 20-26.
[8]	WANG Xialin,KAN Xiu,FAN Yixuan. Classification and Recognition of P300 Event-Related Potential Based on LSTM-Attention Network [J]. Electronic Science and Technology, 2022, 35(12): 10-16.
[9]	YU Runrun,JIANG Xiaoyan,ZHU Kaiying,JIANG Guanghao. Real-Time Image Semantic Segmentation Based on Contextual Attention Mechanism [J]. Electronic Science and Technology, 2022, 35(12): 57-63.
[10]	ZHANG Shisen,SUN Xiankun,YIN Ling,LI Shixi. Design of Text Title Generation Prototype System Based on Neural Network [J]. Electronic Science and Technology, 2021, 34(5): 35-41.
[11]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.
[12]	LI Fupeng,FU Dongxiang. Sentiment Analysis Method of Financial Text Based on Transformer Encoder [J]. Electronic Science and Technology, 2020, 33(9): 10-15.
[13]	TONG Bo,LIU Tao,LIU Chang. Research on Bearing Failure Signal Generation Based on Generative Adversarial Networks [J]. Electronic Science and Technology, 2020, 33(4): 28-34.
[14]	ZHAN Yan,ZHANG Juan. A Domain Adaptive Depth Estimation Method for Structural Perception Loss [J]. Electronic Science and Technology, 2020, 33(12): 12-16.
[15]	ZHENG Meng. Research on Intelligent English Translation Method Based on Improved Attention Mechanism Model [J]. Electronic Science and Technology, 2020, 33(11): 84-87.