一种基于GAN的轻量级水墨画风格迁移模型

doi:10.16180/j.cnki.issn1007-7820.2023.02.012

电子科技 ›› 2023, Vol. 36 ›› Issue (2): 81-86.doi: 10.16180/j.cnki.issn1007-7820.2023.02.012

• • 上一篇

一种基于GAN的轻量级水墨画风格迁移模型

赵晋,李菲菲

上海理工大学上海康复器械工程技术研究中心,上海 200093

收稿日期:2021-08-30 出版日期:2023-02-15 发布日期:2023-01-17
作者简介:赵晋(1995-),男,硕士研究生。研究方向:图像处理与模式识别。。|李菲菲(1970-),女,博士,教授。研究方向:多媒体信息处理、图像处理与模式识别、信息检索等。
基金资助:
上海市高校特聘教授(东方学者)岗位计划(ES2015XX)

A GAN-Based Lightweight Style Transfer Model for Ink Painting

ZHAO Jin,LI Feifei

Shanghai Engineering Research Center of Assistive Devices,University of Shanghai for Science and Technology,Shanghai 200093, China

Received:2021-08-30 Online:2023-02-15 Published:2023-01-17
Supported by:
The Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning(ES2015XX)

摘要/Abstract

摘要：

当前现有的风格迁移方法大多以照片或西方绘画为主。由于中西方画作之间的内在差异,直接应用现有的算法无法生成令人满意的中国水墨画风格迁移的结果。文中基于GAN提出了一种新颖的适用于水墨画的风格迁移方法。该方法结合了AdaIN方法、风格注意力模块和感知损失,可以更准确地学习到水墨画的风格特征,一定程度上解决了水墨图像生成质量不佳的问题。定性分析和定量评估结果表明文中方法性能更好,生成的结果具有更佳的视觉效果。相比于基准方法,文中所提方法减少了约55%的参数量,降低了约60%的训练时间。

关键词: 水墨画, 风格迁移, 生成对抗网络, 注意力机制, ChipGAN, 轻量级, AdaIN, 感知损失

Abstract:

Current style transfer methods are mainly suitable for photos or Western paintings. Due to the inherent differences between Chinese and Western paintings, existing algorithms cannot generate satisfactory results when applied to the style transfer task of Chinese ink paintings. Consequently, a novel style transfer method based on GAN for ink painting is proposed in the study. This method combines AdaIN, style-attention module and perceptual loss to learn the style features of ink painting more accurately, thereby solving the problem that the generated results of ink painting have poor quality. According to the qualitative analysis and quantitative evaluation, experimental results show that this method has better performance and the generated results have higher visual quality. Compared with the baseline, the method reduces the number of parameters by about 55% and the training time by about 60%.

Key words: ink painting, style transfer, generative adversarial networks, attention mechanism, ChipGAN, lightweight, AdaIN, perceptual loss

中图分类号:

TP391

赵晋,李菲菲. 一种基于GAN的轻量级水墨画风格迁移模型[J]. 电子科技, 2023, 36(2): 81-86.

ZHAO Jin,LI Feifei. A GAN-Based Lightweight Style Transfer Model for Ink Painting[J]. Electronic Science and Technology, 2023, 36(2): 81-86.

图/表 5

参考文献 19

[1]	Sheng J, Li Y. Classification of traditional Chinese paintings using a modified embedding algorithm[J]. Journal of Electronic Imaging, 2019, 28(2):1-12.
[2]	Sheng J, Li Y. Style-based classification of Chinese ink and wash paintings[J]. Optical Engineering, 2013, 52(9): 1-9.
[3]	LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444. doi: 10.1038/nature14539
[4]	Gatys L A, Ecker A S, Bethge M. Image style transfer using convolutional neural networks[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[5]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]. San Diego: International Conference on Learning Representations, 2015.
[6]	Huang X, Belongie S. Arbitrary style transfer in real-time with adaptive instance normalization[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision, 2017.
[7]	Goodfellow I, Pouget Abadie J, Mirza M, et al. Generative adversarial nets[C]. Montreal: Annual Conference on Neural Information Processing Systems, 2014.
[8]	Isola P, Zhu J Y, Zhou T H, et al. Image-to-image translation with conditional adversarial networks[C]. Hawaii: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[9]	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. Venice: Proceedings of the IEEE Conference on International Conference on Computer Vision, 2017.
[10]	He B, Gao F, Ma D, et al. ChipGAN: A generative adversarial network for chinese ink wash painting style transfer[C]. Seoul: Proceedings of the Twenty-sixth ACM International Conference on Multimedia, 2018.
[11]	Lee H Y, Tseng H Y, Huang J B, et al. Diverse image-to-image translation via disentangled representations[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018.
[12]	Wang X, Yu K, Wu S, et al. ESRGAN: Enhanced super-resolution generative adversarial networks[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018.
[13]	向晴, 袁健华. 基于多层次判别器的图像翻译模型[J]. 软件, 2020, 41(3):11-17.
	Xiang Qing, Yuan Jianhua. Image-to-image translation based on multi-layer discriminator[J]. Software, 2020, 41(3):11-17.
[14]	佟博, 刘韬, 刘畅. 基于生成对抗网络的轴承失效信号生成研究[J]. 电子科技, 2020, 33(4):28-34.
	Tong Bo, Liu Tao, Liu Chang. Research on bearing failure signal generation based on generative adversarial networks[J]. Electronic Science and Technology, 2020, 33(4):28-34.
[15]	Wang X, Girshick R, Gupta A, et al. Non-local neural networks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[16]	Mao X D, Li Q, Xie H R, et al. Least squares generative adversarial networks[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision, 2017.
[17]	Johnson J, Alahi A, Li F F. Perceptual losses for real time style transfer and super-resolution[C]. Amsterdam: European Conference on Computer Vision, 2016.
[18]	Heusel M, Ramsauer H, Unterthiner T, et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium[C]. Long Beach: Annual Conference on Neural Information Processing Systems, 2017.
[19]	Bińkowski M, Sutherland D J, Arbel M, et al. Demystifying MMD GANs[C]. Vancouver: International Conference on Learning Representations, 2018.

	FID	KID
ChipGAN	203.30	7.61
本文模型	201.00	6.80

	参数量/MB	训练时间/h
ChipGAN	11.378	55.2
本文模型	5.100	21.7

一种基于GAN的轻量级水墨画风格迁移模型

A GAN-Based Lightweight Style Transfer Model for Ink Painting

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 19

相关文章 15

Metrics

本文评价

推荐阅读 10

[1]	左斌,李菲菲. 基于注意力机制和Inf-Net的新冠肺炎图像分割方法[J]. 电子科技, 2023, 36(2): 22-28.
[2]	孙抗,轩旭阳,刘鹏辉,赵来军,龙洁. 小样本下基于CNN-DCGAN的电缆局部放电模式识别方法[J]. 电子科技, 2022, 35(7): 7-13.
[3]	陈劲宏,陈玮,尹钟. 基于改进ExfuseNet模型的街景语义分割[J]. 电子科技, 2022, 35(6): 28-34.
[4]	林潮威,李菲菲,陈虬. 基于深度卷积特征的场景全局与局部表示方法[J]. 电子科技, 2022, 35(4): 20-27.
[5]	纪玲玉,高永彬,赵呈陆,汤先华,徐凯成,徐嘉诚. 基于3D全卷积网络的腹部动脉CTA分割算法[J]. 电子科技, 2022, 35(3): 38-44.
[6]	李辉,王一丞. 基于CNNCIFG-Attention模型的文本情感分类[J]. 电子科技, 2022, 35(2): 46-51.
[7]	王夏霖,阚秀,范艺璇. 基于LSTM-Attention的P300事件相关电位识别分类研究[J]. 电子科技, 2022, 35(12): 10-16.
[8]	于润润,姜晓燕,朱凯赢,蒋光好. 基于上下文注意力机制的实时语义分割[J]. 电子科技, 2022, 35(12): 57-63.
[9]	张伟,刘娜,江洋,李清都. 基于YOLO神经网络的垃圾检测与分类[J]. 电子科技, 2022, 35(10): 45-50.
[10]	张仕森,孙宪坤,尹玲,李世玺. 基于神经网络的文本标题生成原型系统设计[J]. 电子科技, 2021, 34(5): 35-41.
[11]	薛永杰,巨志勇. 基于改进AlexNet的鱼类识别算法[J]. 电子科技, 2021, 34(4): 12-17.
[12]	李福鹏,付东翔. 基于Transformer编码器的金融文本情感分析方法[J]. 电子科技, 2020, 33(9): 10-15.
[13]	佟博,刘韬,刘畅. 基于生成对抗网络的轴承失效信号生成研究[J]. 电子科技, 2020, 33(4): 28-34.
[14]	詹雁,张娟. 一种结构感知损失的域适应深度估计方法[J]. 电子科技, 2020, 33(12): 12-16.
[15]	郑萌. 基于改进注意力机制模型的智能英语翻译方法研究[J]. 电子科技, 2020, 33(11): 84-87.