基于深度学习与显著性的数字图像构图优化

doi:10.16180/j.cnki.issn1007-7820.2021.03.007

Abstract

Abstract:

Image composition is an important factor in the digital image aesthetic quality. Existing computer optimization algorithms have a number of shortcomings, such as visual imbalance and poor integrity. Therefore, this paper proposes a novel composition optimization algorithm based on the deep learning model. This algorithm combines salient features with improved visual balance to achieve composition optimization that is more in line with image aesthetics. The deep convolutional neural network is based on VGG-16, weighting two loss functions and averaging image pixel values. After training, the network could achieve end-to-end saliency regression without any pre-processing or post-processing. Comparative experiments and quantitative verification results demonstrate that compared with the current traditional optimization approaches, the proposed algorithm has obvious advantages in improving visual balance. The processed image has a significantly improved sense of balance, which is more in line with the principles of aesthetic evaluation and visual perception.

Key words: image aesthetic, composition optimization, deep learning, salient feature, end-to-end model, visual balance

CLC Number:

TP391

SHAO Hang,WANG Yongxiong,QIN Yulong. Digital Image Composition Optimization Based on Salient Feature Algorithm[J].Electronic Science and Technology, 2021, 34(3): 36-42.

Figures/Tables 12

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Table 1

Figure 7.

Table 2

Figure 8.

Table 3

Figure 9.

References 31

[1]	Neumann L, Sbert M, Gooch B. Defining computational aesthetics[J]. Computational Aesthetics in Graphics, Visualization and Imaging, 2005(5):13-18.
[2]	王伟凝, 蚁静缄, 贺前华. 可计算图像美学研究进展[J]. 中国图象图形学学报, 2012,17(8):893-901.
	Wang Weining, Yi Jingjian, He Qianhua. Review for computational image aesthetics[J]. Journal of Image and Graphics, 2012,17(8):893-901.
[3]	Deng Y, Loy C, Tang X. Image aesthetic assessment: An experimental survey[J]. IEEE Signal Processing Magazine, 2017,34(4):80-106.
[4]	胡少聪. 基于深度学习的人脸识别方法研究[J]. 电子科技, 2019,32(6):82-86.
	Hu Shaocong. Research on face recognition based on deep learning[J]. Electronic Science and Technology, 2019,32(6):82-86.
[5]	Lu P, Zhang H, Peng X. Aesthetic guided deep regression network for image cropping[J]. Signal Processing Image Communication, 2019,77(6):1-10.
[6]	Kong S, Shen X, Lin Z, et al. Photo aesthetics ranking network with attributes and content adaptation[C]. Amsterdam:Proceedings of European Conference on Computer Vision, 2016.
[7]	Wang W, Shen J. Deep cropping via attention box prediction and aesthetics assessment[C]. Venice: Proceedings of IEEE International Conference on Computer Vision, 2017.
[8]	Zhu J, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. Venice:Proceedings of IEEE International Conference on Computer Vision, 2017.
[9]	Levada A. Closed-form Bayesian image denoising: Improving the adaptive Wiener filter through pairwise Gaussian-Markov random fields[J]. Communication in Statistics Simulation and Computation, 2019(7):1-25.
[10]	Ignatov A, Kobyshev N, Timofte R, et al. DSLR-quality photos on mobile devices with deep convolutional networks[C]. Venice:Proceedings of IEEE International Conference on Computer Vision, 2017.
[11]	Yao L, Suryanarayan P, Qiao M, et al. Oscar:On-site composition and aesthetics feedback through exemplars for photographers[J]. International Journal of Computer Vision, 2012,96(9):353-383. doi: 10.1007/s11263-011-0478-3
[12]	顾婷婷, 郭延文, 殷昆燕. 结合浅景深与构图的图像质量评价[J]. 中国图象图形学学报, 2013,18(5):574-582. doi: 10.11834/jig.20130512
	Gu Tingting, Guo Yanwen, Yin Kunyan. Image quality assessment combining low DoF and composition[J]. Journal of Image and Graphics, 2013,18(5):574-582. doi: 10.11834/jig.20130512
[13]	袁小平, 王岗, 王晔枫, 等. 基于改进卷积神经网络的交通标志识别方法[J]. 电子科技, 2019,32(11):28-32.
	Yuan Xiaoping, Wang Gang, Wang Yefeng, et al. Traffic sign recognition method based on improved convolutional neural network[J]. Electronic Science and Technology, 2019,32(11):28-32.
[14]	凌艳, 陈莹. 多尺度上下文信息增强的显著目标检测全卷积网络[J]. 计算机辅助设计与图形学学报, 2019,31(11):2007-2016.
	Ling Yan, Chen Ying. Salient object detection with multiscale context enhanced fully convolutional network[J]. Journal of Computer-Aided Design & Computer Graphics, 2019,31(11):2007-2016.
[15]	Bhattacharya S, Sukthankar R, Shah M. A framework for photo quality assessment and enhancement based on visual aesthetics[C]. Firenze:Proceedings of ACM International Conference on Multimedia, 2010.
[16]	Jin Y, Wu Q, Liu L. Aesthetic photo composition by optimal crop-and-warp[J]. Computers and Graphics, 2012,36(8):955-965. doi: 10.1016/j.cag.2012.07.007
[17]	Guo Y, Liu M, Gu T, et al. Improving photo composition elegantly:Considering image similarity during composition optimization[J]. Computer Graphics Forum, 2012,31(2):193-202.
[18]	Zhang F, Wang M. Aesthetic image enhancement by dependence-aware object recomposition[J]. IEEE Transactions on Multimedia, 2013,15(7):1480-1490. doi: 10.1109/TMM.2013.2268051
[19]	王伟凝, 刘剑聪, 徐向民, 等. 基于构图规则的图像美学优化[J]. 华南理工大学学报(自然科学版), 2015(5):51-58.
	Wang Weining, Liu Jiancong, Xu Xiangmin, et al. Aesthetic enhancement of images based on photography composition guidelines[J]. Journal of South China University of Technology (Natural Science Edition), 2015(5):51-58.
[20]	熊杨超. 图像美学评价及美学优化研究[D]. 广州:华南理工大学, 2015.
	Xiong Yangchao. Research on image aesthetic assessment and optimizing[D]. Guangzhou: South China University of Technology, 2015.
[21]	Chen J, Bai G, Liang S, et al. Automatic image cropping: A computational complexity study[C]. Las Vegas: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[22]	Chen Y, Huang T, Chang K, et al. Quantitative analysis of automatic image cropping algorithms: A dataset and comparative study[C]. Santa Rosa:Proceedings of IEEE Winter Conference on Applications of Computer Vision, 2017.
[23]	Wang W, Shen J, Ling H. A deep network solution for attention and aesthetics aware photo cropping[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019,41(9):1531-1544. doi: 10.1109/TPAMI.34
[24]	何援军. 透视和透视投影变换—论图形变换和投影的若干问题之三[J]. 计算机辅助设计与图形学学报, 2005,17(4):735-739.
	He Yuanjun. Perspective and its projection transformation[J]. Journal of Computer Aided Design and Computer Graphics, 2005,17(4):735-739.
[25]	Liu T, Sun J, Zheng N, et al. Learning to detect a salient object[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011,33(2):353-367. pmid: 21193811
[26]	Ke Y, Tang X, Jing F. The design of high-level features for photo quality assessment[C]. New York:Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2006.
[27]	Luo W, Wang X, Tang X. Content-based photo quality assessment[J]. IEEE Transactions on Multimedia, 2013(5):1930-1943.
[28]	Murray N, Marchesotti L, Perronnin F. AVA: A large scale database for aesthetic visual analysis[C]. Rhode Island:Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2012.
[29]	Malu G, Bapi R, Indurkhya B. Learning photography aesthetics with deep CNNs[C]. Fort Wayne:Proceedings of Modern Artificial Intelligence and Cognitive Science, 2017.
[30]	Zhu W, Liang S, Wei Y, et al. Saliency optimization from robust background detection[C]. Columbus:Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[31]	Li H, Lu H, Lin Z, et al. Inner and inter label propagation: salient object detection in the wild[J]. IEEE Transactions on Image Processing, 2015,24(4):3176-3186. doi: 10.1109/TIP.83

方法	ℓ₁损失	ℓ₂损失	PSNR	SSIM
wCtr	14.06%	18.34%	10.97	79.93%
LPS	17.01%	18.49%	9.98	75.21%
本文	16.13%	18.68%	9.13	72.81%

方法	ℓ₁损失	ℓ₂损失	PSNR	SSIM
wCtr	8.26%	9.42%	11.62	82.12%
LPS	8.16%	9.74%	12.14	83.24%
本文	6.29%	7.87%	12.69	89.45%

方法	ℓ₁损失	ℓ₂损失	PSNR	SSIM
文献[17]	18.58%	10.21%	19.74	78.04%
文献[19]	22.95%	13.74%	15.65	59.24%
文献[22]	6.63%	8.69%	12.57	85.19%
本文	6.29%	7.87%	12.69	89.45%

Digital Image Composition Optimization Based on Salient Feature Algorithm

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 31

Related Articles 15

Metrics

Comments

Recommended 10

[1]	CHENG Xiaoya,ZHANG Lei. Research on Occluded Face Recognition Method Based on Deep Learning [J]. Electronic Science and Technology, 2022, 35(1): 35-39.
[2]	ZHAO Chong,CHI Mengmeng,CHU Cong,ZHANG Peng. Research on Motion Simulation and Visual Recognition Algorithm of Guide Dog Walking Mechanism [J]. Electronic Science and Technology, 2021, 34(9): 66-72.
[3]	MA Lixin,DOU Chenfei,SONG Chencan,YANG Tianxiao. Insulator Nondestructive Testing Based on Feature Fusion CNN [J]. Electronic Science and Technology, 2021, 34(7): 26-30.
[4]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.
[5]	YAN Chao,SUN Zhanquan,TIAN Engang,ZHAO Yangyang,FAN Xiaoyan. Research Progress of Medical Image Segmentation Based on Deep Learning [J]. Electronic Science and Technology, 2021, 34(2): 7-11.
[6]	CHENG Junhua,ZENG Guohui,LIU Jin. Research on Complex Background Image Classification Method Based on Deep Learning [J]. Electronic Science and Technology, 2020, 33(12): 59-66.
[7]	WU Qian,WU Fei,LUO Lizhi. Human Skeleton-based Action Recognition Based on Deep Learning [J]. Electronic Science and Technology, 2020, 33(11): 79-83.
[8]	DUAN Yujia,JU Ting. Evaluation of Code Review Comments Based on Deep Learning [J]. Electronic Science and Technology, 2020, 33(1): 39-45.
[9]	TONG Zeyou,FENG Shimin,HOU Xiaoqing,DING Enjie. Recognition of Underground Miners’ Rule-Violated Behavior Based on Safety Helmet Detection [J]. Electronic Science and Technology, 2019, 32(9): 26-31.
[10]	HU Shaocong. Research on Face Recognition Based on Deep Learning [J]. Electronic Science and Technology, 2019, 32(6): 82-86.
[11]	JIANG Meng,WANG Ziniu,GAO Jianling. Chinese Word Segmentation Based on Joint Training of Heterogeneous Data [J]. Electronic Science and Technology, 2019, 32(4): 29-33.
[12]	LI Rongrui,SHI Lin,ZHAO Wei. Minority Headdress Recognition Based on Convolutional Neural Network [J]. Electronic Science and Technology, 2019, 32(2): 51-55.
[13]	MENG Xin. Research on Recognition Method of Legal Documents Based on Deep Learning [J]. Electronic Science and Technology, 2019, 32(12): 84-86.
[14]	YUAN Xiaoping,WANG Gang,WANG Yefeng,WANG Zheyuan,SUN Hui. Traffic Sign Recognition Method Based on Improved Convolutional Neural Network [J]. Electronic Science and Technology, 2019, 32(11): 28-32.
[15]	CHENG Lingfei,HE Yang,ZHANG Peiling,LI Yan. Classification Performance of Compressing Dimensionality of Hidden Layer of Deep Neural Network [J]. Electronic Science and Technology, 2019, 32(1): 72-75.