融合注意力机制的弱监督语义分割自激活方法

doi:10.16180/j.cnki.issn1007-7820.2025.04.012

摘要/Abstract

摘要：

弱监督语义分割以类激活图进行训练,但类激活图与真实像素级标签存在较大差距。针对弱监督语义分割类激活图定位信息少、分割结果的轮廓粗糙问题,文中提出基于注意力机制的弱监督语义分割自激活模型。利用仿射变化引入全监督方法中的隐性约束,提取分类网络浅层信息并融合注意力机制。利用增强后的浅层信息细化类激活图的轮廓,根据生成的类激活图对特征图进行自激活,从而生成最终的类激活图。在PASCAL VOC 2012数据集上进行实验,相较于近期先进模型,类激活图的平均交并比提升了1.7%,最终分割结果的平均交并比提升了2.4%。通过对模型进行消融实验验证了各模块的有效性。

关键词: 弱监督方法, 语义分割, 类激活图, 注意力机制, 卷积神经网络, 自激活方法, 仿射变换, 浅层网络, 隐性约束

Abstract:

Weakly supervised semantic segmentation is typically trained by class activation maps, but there is a significant gap between class activation maps and real pixel-level labels. A self-activation model of weakly supervised semantic segmentation based on attention mechanism is proposed to solve the problem that class activation maps of weakly supervised semantic segmentation have little positioning information and rough contour of segmentation results. The implicit constraints in the full supervision method are introduced by affine changes, the shallow information of the classification network is extracted and the attention mechanism is integrated. The enhanced shallow information is used to refine the outline of the class activation diagram, and the feature diagram is self-activated according to the generated class activation diagram, so as to generate the final class activation diagram. Experiments on the PASCAL VOC 2012 data set show a 1.7% improvement in the average crossover ratio of class activation graphs and a 2.4% improvement in the average crossover ratio of final segmentation results compared to recent advanced models. The effectiveness of each module is verified by the ablation experiments.

Key words: weak supervision method, semantic segmentation, class activation map, attention mechanism, convolutional neural network, self-activation method, affine transformation, shallow network, implicit constraint

中图分类号:

TP391

周凯, 于莲芝. 融合注意力机制的弱监督语义分割自激活方法[J]. 电子科技, 2025, 38(4): 80-86.

ZHOU Kai, YU Lianzhi. Self-Activated Learning Method for Weakly Supervised Semantic Segmentation Integrating Attention Mechanism[J]. Electronic Science and Technology, 2025, 38(4): 80-86.

图/表 10

图1

图2

图3

图4

表1

表2

图5

表3

表4

图6

参考文献 21

[1]	Chen L C, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets,atrous convolution and fully connected CRFS[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4):834-848.
[2]	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:801-818.
[3]	Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:2881-2890.
[4]	Cordts M, Omran M, Ramos S, et al. The cityscapes dataset for semantic urban scene understanding[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:3213-3223.
[5]	景庄伟, 管海燕, 彭代峰, 等. 基于深度神经网络的图像语义分割研究综述[J]. 计算机工程, 2020, 46(10):1-17. doi: 10.19678/j.issn.1000-3428.0058018
	Jing Zhuangwei, Guan Haiyan, Peng Daifeng, et al. Survey of research in image semantic segmentation based on deep neural network[J]. Computer Engineering, 2020, 46(10):1-17. doi: 10.19678/j.issn.1000-3428.0058018
[6]	Zhou B, Khosla A, Lapedriza A, et al. Learning deep features for discriminative localization[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:2921-2929.
[7]	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM:Visual explanations from deep networks via gradient-based localization[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision, 2017: 618-626.
[8]	Kolesnikov A, Lampert C H.Seed, expand and constrain: Three principles for weakly-supervised image segmentation[C]. Amsterdam: Proceedings of European Conference on Computer Vision, 2016:695-711.
[9]	Wei Y, Feng J, Liang X, et al. Object region mining with adversarial erasing:A simple classification to semantic segmentation approach[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:1568-1576.
[10]	Ahn J, Kwak S. Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:4981-4990.
[11]	Su Y, Sun R, Lin G, et al. Context decoupling augmentation for weakly supervised semantic segmentation[C]. Montreal: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 7004-7014.
[12]	Lee J, Oh S J, Yun S, et al. Weakly supervised semantic segmentation using out-of-distribution data[C]. New Orleans: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 16897-16906.
[13]	张灿, 陈玮, 尹钟. 基于弱监督宫颈细胞图像的语义分割方法[J]. 电子科技, 2021, 34(12):68-74.
	Zhang Can, Chen Wei, Yin Zhong. Semantic segmentation of cervical cell image based on weak supervision[J]. Electronic Science and Technology, 2021, 34(12):68-74.
[14]	Wang Y, Zhang J, Kan M, et al. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation[C]. Seattle: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020:12275-12284.
[15]	Woo S, Park J, Lee J Y, et al. CBAM:Convolutional block attention module[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:3-19.
[16]	Chen Z, Wang T, Wu X, et al. Class re-activation maps for weakly-supervised semantic segmentation[C]. New Orleans: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022:969-978.
[17]	Everingham M, Van Gool L, Williams C K I, et al. The pascal visual object classes challenge[J]. International Journal of Computer Vision, 2010, 88(2):303-338.
[18]	Hariharan B, Arbeláez P, Bourdev L, et al. Semantic contours from inverse detectors[C]. Barcelona: Proceedings of the IEEE International Conference on Computer Vision, 2011:991-998.
[19]	Ahn J, Cho S, Kwak S. Weakly supervised learning of instance segmentation with inter-pixel relations[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 2209-2218.
[20]	Zhang B, Xiao J, Jiao J, et al. Affinity attention graph neural network for weakly supervised semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(11):8082-8096.
[21]	Ru L, Zhan Y, Yu B, et al. Learning affinity from attention: End-to-end weakly-supervised semantic segmentation with transformers[C]. New Orleans: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022:16846-16855.

基准	仿射变换	特征增强	轮廓细化	自激活	mIoU/%
√	-	-	-	-	48.4
√	-	√	-	-	50.4
√	√	√	-	-	51.6
√	√	√	√	-	52.3
√	√	√	√	√	56.5

方法	mIoU/%
CAM	48.3
CDA	50.8
ReCAM	54.8
SEAM	55.4
本文方法	56.5

方法	mIoU/%
IRNet	66.5
CDA	67.7
本文方法	71.2

方法	主干网络	mIoU/%
SEC	VGG16	50.7
IRNet	ResNet50	63.5
CDA	ResNet50	65.8
AffinityNet	ResNet38	61.7
SEAM	ResNet38	64.5
A²GNN^[20]	ResNet101	66.8
AFA^[21]	MiT-B1	66.0
ReCAM	ResNet5068.5	68.5
本文方法	ResNet50	70.9