基于渐进结构感受野和全局注意力的显著性检测

doi:10.16180/j.cnki.issn1007-7820.2021.01.005

摘要/Abstract

摘要：

当前的显著性检测算法在复杂场景下难以分割出完整显著性区域以及锐利的边缘细节。针对这一问题,文中提出了一种新颖的特征融合算法。该方法利用全卷积神经网络获取多个层次粗糙的初始特征并结合特征金字塔结构对其深度解析。设计渐进结构感受野模块将特征转换至不同尺度的空间进行优化,实现特征的渐进融合与传递,有选择性地增强显著性区域。采用全局注意力机制消除背景噪声并建立显著性像素之间的长距离依赖,以提高显著性区域的有效性,突出显著性目标,再通过学习融合个层次特征得到显著图。综合实验表明,在绝对误差减小的情况下,F-measure指标远超出其他7种主流方法。所提的显著性模型综合了全卷积神经网络和特征金字塔结构的优点,结合文中设计的渐进结构感受野和全局注意力机制,使得显著图更接近真值图。

关键词: 显著性检测, 全卷积神经网络, 特征金字塔, 渐进结构感受野, 全局注意力, F-measure指标

Abstract:

In view of the current deficiencies that the previous saliency detection algorithms are difficult to segment the complete salient region and sharp edge details in complex scenes, a novel feature fusion of saliency detection model is proposed in this paper. The proposed algorithm utilizes full convolution neural network to obtain the initial features of multi-level roughness and combines the feature pyramid structure to analyze its depth. In order to realize the gradual fusion and transmission of features, the progressive structural receptive field module is designed to transform features to different scales of space for optimization. The global attention mechanism is used to eliminate the background noise and establish the long-distance dependence between the saliency pixels, so as to improve the effectiveness of the saliency region, highlight the saliency region, and then obtains the saliency map by learning and fusing the hierarchical features. The comprehensive experiment show that the F-measure index is far beyond the other seven mainstream methods when the absolute error is reduced. The proposed saliency model combines the advantages of full convolution neural network and feature pyramid structure, and combines the gradual structure receptive field and global attention mechanism designed in this study to make the saliency map closer to the truth map.

Key words: saliency detection, fully convolutional networks, feature pyramid, progressive structural receptive field, global attention, F-measure index

中图分类号:

TP391

董波,周燕,王永雄. 基于渐进结构感受野和全局注意力的显著性检测[J]. 电子科技, 2021, 34(1): 23-30.

DONG Bo,ZHOU Yan,WANG Yongxiong. Saliency Detection by Progressive Structural Receptive Field and Global Attention[J]. Electronic Science and Technology, 2021, 34(1): 23-30.

图/表 7

图1

图2

图3

图4

表1

图5

表2

参考文献 31

[1]	Ren Z X, Gao S H, Chia L, et al. Region-based saliency detection and its application in object recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2013,24(5):769-779. doi: 10.1109/TCSVT.2013.2280096
[2]	Gao Y, Wang M, Tao D C, et al. 3-D object retrieval and recognition with hypergraph analysis[J]. IEEE Transactions on Image Processing, 2012,21(9):4290-4303. doi: 10.1109/TIP.2012.2199502 pmid: 22614650
[3]	He J F, Feng J Y, Liu X L, et al. Mobile product search with bag of hash bits and boundary reranking[C]. Providence:IEEE Conference on Computer Vision and Pattern Recognition, 2012.
[4]	Borji A, Frintrop S, Sihite D, et al. Adaptive object tracking by learning background context[C]. Providence:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2012.
[5]	Mahadevan V, Vasconcelos N. Saliency-based discriminant tracking[C]. Miami:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2009.
[6]	Zhao R, Quyang W L, Wang X G. Unsupervisedsalience learning for person re-identification[C]. Portland:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013.
[7]	Wang L J, Lu H C, Wang Y F, et al. Learning to detect salient objects with image-level supervision[C]. Honolulu:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[8]	郭迎春, 袁浩杰, 吴鹏, 等. 基于Local特征和Regional特征的图像显著性检测[J]. 自动化学报, 2013,39(8):1214-1224. doi: 10.3724/SP.J.1004.2013.01214
	Guo Yingchun, Yuan Haojie, Wu Peng, et al. Image saliency detection based on local and regional features[J]. Acta Automatica Sinica, 2013,39(8):1214-1224. doi: 10.3724/SP.J.1004.2013.01214
[9]	刘甜甜. 基于稀疏和低秩表示的显著性目标检测[J]. 电子科技, 2015,28(2):112-115.
	Liu Tiantian. Spare and low rank based saliency detection[J]. Electronic Science and Technology, 2015,28(2):112-115.
[10]	Cheng M M, Zhang G X, Mitra N J, et al. Global contrast based salient region detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014,37(3):569-582. pmid: 26353262
[11]	张昆. 基于全局和局部信息融合的图像显著性检测研究[J]. 电子科技, 2018,31(5):84-88.
	Zhang Kun. Research on image saliency detection based on global and local information fusion[J]. Electronic Science and Technology, 2018,31(5):84-88.
[12]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]. Columbus:Proceedings of Computer Science, 2014.
[13]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[14]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]. Boston:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[15]	Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation[C]. Boston:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[16]	Xie S N, Tu Z W. Holistically-nested edge detection[C]. Boston:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[17]	Hou Q B, Cheng M M, Hu X W, et al. Deeply supervised salient object detection with short connections[C]. Hawaii:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[18]	Zhang P P, Wang D, Lu H C, et al. Amulet: Aggregating multi-level convolutional features for salient object detection[C]. Hawaii:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[19]	Krähenbühl P, Koltun V. Efficient inference in fully connected crfs with gaussian edge potentials[C]. Colorado Springs:Proceedings of the Neural Information Processing Systems, 2011.
[20]	钱生, 陈宗海, 林名强, 等. 基于条件随机场和图像分割的显著性检测[J]. 自动化学报, 2015,41(4):711-724.
	Qian Sheng, Chen Zonghai, Lin Mingqiang, et al. Saliency detection based on conditional random field and image segmentation[J]. Acta Automation, 2015,41(4):711-724.
[21]	Liu N, Han J W, Yang M. PiCANet: Learning pixel-wise contextual attention for saliency detection[C]. Salt Lake:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018
[22]	Md Amirul I, Mrigank R, Shujon N, et al. Gated feedback refinement network for dense image labeling[C]. Hawaii:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[23]	Zhang X N, Wang T T, Qi J Q, et al. Progressive attention guided recurrent network for salient object detection[C]. Salt Lake:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[24]	Liu N, Han J W. Dhsnet:deep hierarchical saliency network for salient object detection[C]. Las Vegas:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[25]	Xi X Y, Luo Y K, Wang P, et al. Salient object detection based on an efficient end-to-end saliency regression network[J]. Neurocomputing, 2019,323(1):265-276.
[26]	Yan Q, Xu L, Shi J P, et al. Hierarchical saliency detection[C]. Portland:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013.
[27]	Li Y, Hou X D, Koch C, et al. The secrets of salient object segmentation[C]. Columbus:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[28]	Li G B, Yu Y Z. Visual saliency based on multiscale deep features[C]. Boston:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[29]	Wang L, Wang L, Lu H, et al. Saliency detection with recurrent fully convolutional networks[C]. Las Vegas:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[30]	Zhang P, Wang D, Lu H, et al. Learning uncertain convolutional features for accurate saliency detection[C]. Hawaii:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[31]	Luo Z, Mishra A, Achkar A, et al. Non-local deep features for salient object detection[C]. Hawaii:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.

Data	ECSSD			PASCAL			DUTS-TE			DUT-OMRON			HKUIS
Model	MaxF	AvgF	MAE	MaxF	AvgF	MAE	MaxF	AvgF	MAE	MaxF	AvgF	MAE	MaxF	AvgF	MAE
MDF	0.832	0.705	0.192	0.782	0.579	0.165	0.711	0.509	0.114	0.694	0.565	0.092	-	-	-
RFCN	0.898	0.727	0.095	0.850	0.671	0.132	0.783	0.587	0.090	0.738	0.562	0.095	0.898	0.718	0.080
UCF	0.911	0.789	0.078	0.846	0.709	0.128	0.771	0.588	0.117	0.735	0.565	0.132	0.886	0.751	0.074
NLDF	0.915	0.886	0.051	0.840	0.792	0.083	0.830	0.759	0.055	0.759	0.694	0.071	0.908	0.871	0.041
Amulet	0.922	0.881	0.057	0.839	0.780	0.095	0.832	0.738	0.062	0.791	0.699	0.072	0.909	0.863	0.047
PAGR	0.931	0.894	0.061	0.851	0.803	0.092	0.855	0.788	0.055	0.771	0.711	0.072	0.918	0.886	0.048
PiCANet	0.928	0.885	0.046	0.862	0.796	0.076	0.851	0.749	0.054	0.794	0.710	0.068	0.921	0.870	0.042
本文算法	0.936	0.921	0.040	0.877	0.864	0.075	0.865	0.843	0.043	0.794	0.775	0.057	0.927	0.909	0.033

渐进结构感受野	全局注意力机制	分层式学习融合	DUTS-TE
渐进结构感受野	全局注意力机制	分层式学习融合	MaxF	AvgF	MAE
√			0.860	0.815	0.050
	√		0.611	0.544	0.126
√	√		0.858	0.840	0.047
√	√	√	0.865	0.843	0.043