基于上下文注意力机制的实时语义分割

doi:10.16180/j.cnki.issn1007-7820.2022.12.008

摘要/Abstract

摘要：

针对实时语义分割网络提取的特征缺少上下文信息,易造成分割结果出现类内不一致和类间不可区分的问题,文中提出了轻量级的自适应通道注意力模块和空间注意力模块。自适应通道注意力模块使用深度分离卷积对通道层面的特征依赖关系进行建模,自适应地调整通道卷积核大小,强化高层特征的上下文表征能力,加强了分割结果的类内一致性。空间注意力模块使用分组卷积,以较小的计算量获得较大的特征信息流动区域,在空间层面加强特征的上下文联系,增强特征的空间细节信息,加强了分割结果的类间可区分性。在Cityscapes数据集上的测试与分析表明,轻量级上下文注意力机制获得了71.5%的mIoU。

关键词: 注意力机制, 语义分割, 多分支结构, 信息融合, 深度学习, 分组卷积, 类间差别, 类内一致

Abstract:

To address the problem that feature extracted by real time semantic segmentation model lacks contextual information, which causese inconsistent segmentation results between and within classes, a lightweight adaptive spatial attention model and channel attention model are proposed in this study. The adaptive channel attention module uses depthwise separable convolution to model the channel-level feature dependencies, adaptively adjusts the channel convolution kernel size, strengthens the contextual representation ability of high-level features, and enhances the intra-class consistency of segmentation results. The spatial attention module uses grouped convolution to obtain a larger flow area of feature information with a small amount of calculation, strengthens the contextual connection of features at the spatial level, enhances the spatial detail information of features, and enhances the inter-class distinguishability of segmentation results. Testing and analysis on the Cityscapes data set show that the lightweight contextual attention mechanism achieves 71.5% mIoU.

Key words: attention mechanism, semantic segmentation, multi-branch structure, information fusion, deep learning, grouped convolution, inter-class variation, intra-class consistency

中图分类号:

TP391

于润润,姜晓燕,朱凯赢,蒋光好. 基于上下文注意力机制的实时语义分割[J]. 电子科技, 2022, 35(12): 57-63.

YU Runrun,JIANG Xiaoyan,ZHU Kaiying,JIANG Guanghao. Real-Time Image Semantic Segmentation Based on Contextual Attention Mechanism[J]. Electronic Science and Technology, 2022, 35(12): 57-63.

图/表 11

图1

图2

图3

图4

表1

图5

表2

图6

图7

表3

图8

参考文献 18

[1]	吴展翔, 甘屹, 孙福佳. 基于STL模型的模具型腔分割算法研究及应用[J]. 电子科技, 2019, 32(7):56-59.
	Wu Zhanxiang, Gan Yi, Sun Fujia. Research and application of mold cavity segmentation algorithm based on STL model[J]. Electronic Science and Technology, 2019, 32(7):56-59.
[2]	He W, Wu M, Liang M, et al. CAP: Context-aware pruning for semantic segmentation[C]. Paris: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021.
[3]	Zhao H, Qi X, Shen X, et al. ICNet for real-time semantic segmentation on high-resolution images[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018.
[4]	Yu C, Wang J, Peng C, et al. Bisenet: Bilateral segmentation network for real-time semantic segmentation[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018.
[5]	Yu C, Wang J, Peng C, et al. Learning a discriminative feature network for semantic segmentation[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[6]	Zhong Z, Lin Z Q, Bidart R, et al. Squeeze-and-attention networks for semantic segmentation[C]. Seattle: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
[7]	Fu J, Liu J, Tian H, et al. Dual attention network for scene segmentation[C]. Long Beach: Proceedings of the IEEE/ CVF Conference on Computer Vision and Pattern Recognition, 2019.
[8]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Munich: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[9]	Wang Q L, Wu B G, Zhu P F, et al. ECA-Net: Efficient channel attention for deep convolutional neural networks[C]. Seattle: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[10]	Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[11]	Brostow G J, Fauqueur J, Cipolla R. Semantic object classes in video: A high-definition ground truth database[J]. Pattern Recognition Letters, 2009, 30(2):88-97. doi: 10.1016/j.patrec.2008.04.005
[12]	Cordts M, Omran M, Ramos S, et al. The cityscapes dataset for semantic urban scene understanding[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[13]	Chen Y, Li J, Xiao H, et al. Dual path networks[C]. Barcelona: Proceedings of the Thirty-first International Conference on Neural Information Processing Systems, 2017.
[14]	Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[C]. San Diego: Proceedings of the International Conference on Learning Representations, 2015.
[15]	Li H, Xiong P, Fan H, et al. Dfanet: Deep feature aggregation for real-time semantic segmentation[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
[16]	Poudel R P K, Liwicki S, Cipolla R. Fast-SCNN: Fast semantic segmentation network[C]. Cardiff: Proceedings of the Thirtieth British Machine Vision Conference, 2019.
[17]	Pohlen T, Hermans A, Mathias M, et al. Full-resolution residual networks for semantic segmentation in street scenes[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[18]	Lin P, Sun P, Cheng G, et al. Graph-guided architecture search for real-time semantic segmentation[C]. Seattle: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.

方法	时间/ms	FPS	mIoU/%
DPN^[13]	830	1.2	60.1
DeepLab^[14]	203	4.9	61.6
PSPNet50^[10]	185	5.4	69.1
DFANet A^[15]	8	125.0	64.7
ICNet^[3]	36	27.8	67.1
Ours	35	28.6	68.2

方法	DR	FPS	mIoU/%
DeepLab^[14]	yes	0.025	63.1
Fast-SCNN^[16]	yes	120.000	68.0
FRRN^[17]	yes	2.100	71.8
DFANet A^[14]	no	100.000	71.3
GAS^[18]	yes	111.000	70.5
ICNet^[3]	no	30.300	69.5
Ours	no	33.300	71.5

方法	mIoU/%
ICNet^[3](baseline)	69.5
DCFE-Net	67.5
DCFE-Net + LACAM	69.1
DCFE-Net + LSAM	68.6
DCFE-Net + LACAM + LSAM	71.5