基于深度学习与显著性的数字图像构图优化

doi:10.16180/j.cnki.issn1007-7820.2021.03.007

Abstract

Abstract:

Image composition is an important factor in the digital image aesthetic quality. Existing computer optimization algorithms have a number of shortcomings, such as visual imbalance and poor integrity. Therefore, this paper proposes a novel composition optimization algorithm based on the deep learning model. This algorithm combines salient features with improved visual balance to achieve composition optimization that is more in line with image aesthetics. The deep convolutional neural network is based on VGG-16, weighting two loss functions and averaging image pixel values. After training, the network could achieve end-to-end saliency regression without any pre-processing or post-processing. Comparative experiments and quantitative verification results demonstrate that compared with the current traditional optimization approaches, the proposed algorithm has obvious advantages in improving visual balance. The processed image has a significantly improved sense of balance, which is more in line with the principles of aesthetic evaluation and visual perception.

Key words: image aesthetic, composition optimization, deep learning, salient feature, end-to-end model, visual balance

CLC Number:

TP391

SHAO Hang,WANG Yongxiong,QIN Yulong. Digital Image Composition Optimization Based on Salient Feature Algorithm[J].Electronic Science and Technology, 2021, 34(3): 36-42.

Figures/Tables 12

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Table 1

Figure 7.

Table 2

Figure 8.

Table 3

Figure 9.

References 31

[1]	Neumann L, Sbert M, Gooch B. Defining computational aesthetics[J]. Computational Aesthetics in Graphics, Visualization and Imaging, 2005(5):13-18.
[2]	王伟凝, 蚁静缄, 贺前华. 可计算图像美学研究进展[J]. 中国图象图形学学报, 2012,17(8):893-901.
	Wang Weining, Yi Jingjian, He Qianhua. Review for computational image aesthetics[J]. Journal of Image and Graphics, 2012,17(8):893-901.
[3]	Deng Y, Loy C, Tang X. Image aesthetic assessment: An experimental survey[J]. IEEE Signal Processing Magazine, 2017,34(4):80-106.
[4]	胡少聪. 基于深度学习的人脸识别方法研究[J]. 电子科技, 2019,32(6):82-86.
	Hu Shaocong. Research on face recognition based on deep learning[J]. Electronic Science and Technology, 2019,32(6):82-86.
[5]	Lu P, Zhang H, Peng X. Aesthetic guided deep regression network for image cropping[J]. Signal Processing Image Communication, 2019,77(6):1-10.
[6]	Kong S, Shen X, Lin Z, et al. Photo aesthetics ranking network with attributes and content adaptation[C]. Amsterdam:Proceedings of European Conference on Computer Vision, 2016.
[7]	Wang W, Shen J. Deep cropping via attention box prediction and aesthetics assessment[C]. Venice: Proceedings of IEEE International Conference on Computer Vision, 2017.
[8]	Zhu J, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]. Venice:Proceedings of IEEE International Conference on Computer Vision, 2017.
[9]	Levada A. Closed-form Bayesian image denoising: Improving the adaptive Wiener filter through pairwise Gaussian-Markov random fields[J]. Communication in Statistics Simulation and Computation, 2019(7):1-25.
[10]	Ignatov A, Kobyshev N, Timofte R, et al. DSLR-quality photos on mobile devices with deep convolutional networks[C]. Venice:Proceedings of IEEE International Conference on Computer Vision, 2017.
[11]	Yao L, Suryanarayan P, Qiao M, et al. Oscar:On-site composition and aesthetics feedback through exemplars for photographers[J]. International Journal of Computer Vision, 2012,96(9):353-383. doi: 10.1007/s11263-011-0478-3
[12]	顾婷婷, 郭延文, 殷昆燕. 结合浅景深与构图的图像质量评价[J]. 中国图象图形学学报, 2013,18(5):574-582. doi: 10.11834/jig.20130512
	Gu Tingting, Guo Yanwen, Yin Kunyan. Image quality assessment combining low DoF and composition[J]. Journal of Image and Graphics, 2013,18(5):574-582. doi: 10.11834/jig.20130512
[13]	袁小平, 王岗, 王晔枫, 等. 基于改进卷积神经网络的交通标志识别方法[J]. 电子科技, 2019,32(11):28-32.
	Yuan Xiaoping, Wang Gang, Wang Yefeng, et al. Traffic sign recognition method based on improved convolutional neural network[J]. Electronic Science and Technology, 2019,32(11):28-32.
[14]	凌艳, 陈莹. 多尺度上下文信息增强的显著目标检测全卷积网络[J]. 计算机辅助设计与图形学学报, 2019,31(11):2007-2016.
	Ling Yan, Chen Ying. Salient object detection with multiscale context enhanced fully convolutional network[J]. Journal of Computer-Aided Design & Computer Graphics, 2019,31(11):2007-2016.
[15]	Bhattacharya S, Sukthankar R, Shah M. A framework for photo quality assessment and enhancement based on visual aesthetics[C]. Firenze:Proceedings of ACM International Conference on Multimedia, 2010.
[16]	Jin Y, Wu Q, Liu L. Aesthetic photo composition by optimal crop-and-warp[J]. Computers and Graphics, 2012,36(8):955-965. doi: 10.1016/j.cag.2012.07.007
[17]	Guo Y, Liu M, Gu T, et al. Improving photo composition elegantly:Considering image similarity during composition optimization[J]. Computer Graphics Forum, 2012,31(2):193-202.
[18]	Zhang F, Wang M. Aesthetic image enhancement by dependence-aware object recomposition[J]. IEEE Transactions on Multimedia, 2013,15(7):1480-1490. doi: 10.1109/TMM.2013.2268051
[19]	王伟凝, 刘剑聪, 徐向民, 等. 基于构图规则的图像美学优化[J]. 华南理工大学学报(自然科学版), 2015(5):51-58.
	Wang Weining, Liu Jiancong, Xu Xiangmin, et al. Aesthetic enhancement of images based on photography composition guidelines[J]. Journal of South China University of Technology (Natural Science Edition), 2015(5):51-58.
[20]	熊杨超. 图像美学评价及美学优化研究[D]. 广州:华南理工大学, 2015.
	Xiong Yangchao. Research on image aesthetic assessment and optimizing[D]. Guangzhou: South China University of Technology, 2015.
[21]	Chen J, Bai G, Liang S, et al. Automatic image cropping: A computational complexity study[C]. Las Vegas: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[22]	Chen Y, Huang T, Chang K, et al. Quantitative analysis of automatic image cropping algorithms: A dataset and comparative study[C]. Santa Rosa:Proceedings of IEEE Winter Conference on Applications of Computer Vision, 2017.
[23]	Wang W, Shen J, Ling H. A deep network solution for attention and aesthetics aware photo cropping[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019,41(9):1531-1544. doi: 10.1109/TPAMI.34
[24]	何援军. 透视和透视投影变换—论图形变换和投影的若干问题之三[J]. 计算机辅助设计与图形学学报, 2005,17(4):735-739.
	He Yuanjun. Perspective and its projection transformation[J]. Journal of Computer Aided Design and Computer Graphics, 2005,17(4):735-739.
[25]	Liu T, Sun J, Zheng N, et al. Learning to detect a salient object[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011,33(2):353-367. pmid: 21193811
[26]	Ke Y, Tang X, Jing F. The design of high-level features for photo quality assessment[C]. New York:Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2006.
[27]	Luo W, Wang X, Tang X. Content-based photo quality assessment[J]. IEEE Transactions on Multimedia, 2013(5):1930-1943.
[28]	Murray N, Marchesotti L, Perronnin F. AVA: A large scale database for aesthetic visual analysis[C]. Rhode Island:Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2012.
[29]	Malu G, Bapi R, Indurkhya B. Learning photography aesthetics with deep CNNs[C]. Fort Wayne:Proceedings of Modern Artificial Intelligence and Cognitive Science, 2017.
[30]	Zhu W, Liang S, Wei Y, et al. Saliency optimization from robust background detection[C]. Columbus:Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[31]	Li H, Lu H, Lin Z, et al. Inner and inter label propagation: salient object detection in the wild[J]. IEEE Transactions on Image Processing, 2015,24(4):3176-3186. doi: 10.1109/TIP.83

方法	ℓ₁损失	ℓ₂损失	PSNR	SSIM
wCtr	14.06%	18.34%	10.97	79.93%
LPS	17.01%	18.49%	9.98	75.21%
本文	16.13%	18.68%	9.13	72.81%

方法	ℓ₁损失	ℓ₂损失	PSNR	SSIM
wCtr	8.26%	9.42%	11.62	82.12%
LPS	8.16%	9.74%	12.14	83.24%
本文	6.29%	7.87%	12.69	89.45%

方法	ℓ₁损失	ℓ₂损失	PSNR	SSIM
文献[17]	18.58%	10.21%	19.74	78.04%
文献[19]	22.95%	13.74%	15.65	59.24%
文献[22]	6.63%	8.69%	12.57	85.19%
本文	6.29%	7.87%	12.69	89.45%

Digital Image Composition Optimization Based on Salient Feature Algorithm

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 31

Related Articles 15

Metrics

Comments

Recommended 0

[1]	WANG Ziyi, CHEN Shiping. Self-Supervised Network Intrusion Detection Model Based on Graph Contrastive Learning [J]. Electronic Science and Technology, 2025, 38(3): 22-31.
[2]	ZHAO Yunfei, XUE Cunjin. A Remote Sensing Image Water Extraction Method by Combining Atrous Convolution and Pooling Models [J]. Electronic Science and Technology, 2025, 38(3): 40-46.
[3]	ZHENG Fangliang, WANG Yannian, LIAN Jihong, RUAN Pei. Face Image Super-Resolution Reconstruction Based on Conditional Priori Swin Transformer [J]. Electronic Science and Technology, 2025, 38(2): 35-41.
[4]	LAI Ying, JU Zhiyong, YE Yuxin. A Vehicle Detection Algorithm Based on Improved YOLOv4 [J]. Electronic Science and Technology, 2025, 38(1): 81-87.
[5]	HE Zhiqiang, SUN Zhanquan. Swin-Transformer-Based Carotid Ultrasound Image Plaque Segmentation [J]. Electronic Science and Technology, 2024, 37(9): 48-56.
[6]	HE Xing, HUANG Yongming, ZHU Yong. Pavement Pothole Detection Method Based on Improved YOLOv5 [J]. Electronic Science and Technology, 2024, 37(7): 53-59.
[7]	YE Yuxin, JU Zhiyong, LAI Ying. Traffic Sign Detection Algorithm Incorporating Receptive Field Enhancement Module and Attention Mechanism [J]. Electronic Science and Technology, 2024, 37(6): 8-16.
[8]	YUAN Ao, QI Jinpeng, JIA Can, XUE Yuxin, GUO Yangyang. A Classification Method of Time Series Pathological Data Segments Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(6): 84-91.
[9]	ZHU Zihao, SONG Yan. Lightweight Capsule Network Fusing Attention and Capsule Pooling [J]. Electronic Science and Technology, 2024, 37(5): 1-8.
[10]	XIA Rongcheng, LIU Deer. Vehicle Detection and Analysis in Urban Waterlogging Area Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(5): 18-24.
[11]	MA Wenjie, ZHANG Xuanxiong. Research on Blind Roads and Obstacle Recognition Algorithm Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(3): 75-83.
[12]	CUI Jingnan, HUANG Chunyan, LI Yanling. Improvement of YOLOv5s Algorithm for Steel Surface Defect Detection [J]. Electronic Science and Technology, 2024, 37(12): 48-55.
[13]	LIANG Shuifen, GUO Zhiyong. Advancements of Deep Learning in Imaging Diagnosis of Kidney Stone [J]. Electronic Science and Technology, 2024, 37(12): 67-72.
[14]	LI Yunpeng, XI Zhihong. Dynamic Face Recognition System Design Based on RetinaFace and FaceNet [J]. Electronic Science and Technology, 2024, 37(12): 79-86.
[15]	LIAN Yue, LIU Deer. Outdoor Navigation for Blind People Dynamic Obstacle Target RGB-D Visual Perception and Detection [J]. Electronic Science and Technology, 2024, 37(12): 9-16.