融合注意力机制的弱监督语义分割自激活方法

doi:10.16180/j.cnki.issn1007-7820.2025.04.012

Abstract

Abstract:

Weakly supervised semantic segmentation is typically trained by class activation maps, but there is a significant gap between class activation maps and real pixel-level labels. A self-activation model of weakly supervised semantic segmentation based on attention mechanism is proposed to solve the problem that class activation maps of weakly supervised semantic segmentation have little positioning information and rough contour of segmentation results. The implicit constraints in the full supervision method are introduced by affine changes, the shallow information of the classification network is extracted and the attention mechanism is integrated. The enhanced shallow information is used to refine the outline of the class activation diagram, and the feature diagram is self-activated according to the generated class activation diagram, so as to generate the final class activation diagram. Experiments on the PASCAL VOC 2012 data set show a 1.7% improvement in the average crossover ratio of class activation graphs and a 2.4% improvement in the average crossover ratio of final segmentation results compared to recent advanced models. The effectiveness of each module is verified by the ablation experiments.

Key words: weak supervision method, semantic segmentation, class activation map, attention mechanism, convolutional neural network, self-activation method, affine transformation, shallow network, implicit constraint

CLC Number:

TP391

ZHOU Kai, YU Lianzhi. Self-Activated Learning Method for Weakly Supervised Semantic Segmentation Integrating Attention Mechanism[J].Electronic Science and Technology, 2025, 38(4): 80-86.

Figures/Tables 10

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Table 1.

Table 2.

Figure5.

Table 3.

Table 4.

Figure 6.

References 21

[1]	Chen L C, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets,atrous convolution and fully connected CRFS[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4):834-848.
[2]	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:801-818.
[3]	Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:2881-2890.
[4]	Cordts M, Omran M, Ramos S, et al. The cityscapes dataset for semantic urban scene understanding[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:3213-3223.
[5]	景庄伟, 管海燕, 彭代峰, 等. 基于深度神经网络的图像语义分割研究综述[J]. 计算机工程, 2020, 46(10):1-17. doi: 10.19678/j.issn.1000-3428.0058018
	Jing Zhuangwei, Guan Haiyan, Peng Daifeng, et al. Survey of research in image semantic segmentation based on deep neural network[J]. Computer Engineering, 2020, 46(10):1-17. doi: 10.19678/j.issn.1000-3428.0058018
[6]	Zhou B, Khosla A, Lapedriza A, et al. Learning deep features for discriminative localization[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:2921-2929.
[7]	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM:Visual explanations from deep networks via gradient-based localization[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision, 2017: 618-626.
[8]	Kolesnikov A, Lampert C H.Seed, expand and constrain: Three principles for weakly-supervised image segmentation[C]. Amsterdam: Proceedings of European Conference on Computer Vision, 2016:695-711.
[9]	Wei Y, Feng J, Liang X, et al. Object region mining with adversarial erasing:A simple classification to semantic segmentation approach[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:1568-1576.
[10]	Ahn J, Kwak S. Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:4981-4990.
[11]	Su Y, Sun R, Lin G, et al. Context decoupling augmentation for weakly supervised semantic segmentation[C]. Montreal: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 7004-7014.
[12]	Lee J, Oh S J, Yun S, et al. Weakly supervised semantic segmentation using out-of-distribution data[C]. New Orleans: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 16897-16906.
[13]	张灿, 陈玮, 尹钟. 基于弱监督宫颈细胞图像的语义分割方法[J]. 电子科技, 2021, 34(12):68-74.
	Zhang Can, Chen Wei, Yin Zhong. Semantic segmentation of cervical cell image based on weak supervision[J]. Electronic Science and Technology, 2021, 34(12):68-74.
[14]	Wang Y, Zhang J, Kan M, et al. Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation[C]. Seattle: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020:12275-12284.
[15]	Woo S, Park J, Lee J Y, et al. CBAM:Convolutional block attention module[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:3-19.
[16]	Chen Z, Wang T, Wu X, et al. Class re-activation maps for weakly-supervised semantic segmentation[C]. New Orleans: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022:969-978.
[17]	Everingham M, Van Gool L, Williams C K I, et al. The pascal visual object classes challenge[J]. International Journal of Computer Vision, 2010, 88(2):303-338.
[18]	Hariharan B, Arbeláez P, Bourdev L, et al. Semantic contours from inverse detectors[C]. Barcelona: Proceedings of the IEEE International Conference on Computer Vision, 2011:991-998.
[19]	Ahn J, Cho S, Kwak S. Weakly supervised learning of instance segmentation with inter-pixel relations[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 2209-2218.
[20]	Zhang B, Xiao J, Jiao J, et al. Affinity attention graph neural network for weakly supervised semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(11):8082-8096.
[21]	Ru L, Zhan Y, Yu B, et al. Learning affinity from attention: End-to-end weakly-supervised semantic segmentation with transformers[C]. New Orleans: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022:16846-16855.

基准	仿射变换	特征增强	轮廓细化	自激活	mIoU/%
√	-	-	-	-	48.4
√	-	√	-	-	50.4
√	√	√	-	-	51.6
√	√	√	√	-	52.3
√	√	√	√	√	56.5

方法	mIoU/%
CAM	48.3
CDA	50.8
ReCAM	54.8
SEAM	55.4
本文方法	56.5

方法	mIoU/%
IRNet	66.5
CDA	67.7
本文方法	71.2

方法	主干网络	mIoU/%
SEC	VGG16	50.7
IRNet	ResNet50	63.5
CDA	ResNet50	65.8
AffinityNet	ResNet38	61.7
SEAM	ResNet38	64.5
A²GNN^[20]	ResNet101	66.8
AFA^[21]	MiT-B1	66.0
ReCAM	ResNet5068.5	68.5
本文方法	ResNet50	70.9

Self-Activated Learning Method for Weakly Supervised Semantic Segmentation Integrating Attention Mechanism

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 21

Related Articles 15

Metrics

Comments

Recommended 0

[1]	FU Peng, SONG Xiaoxia. Seismic Data Denoising Method Based on CycleGAN [J]. Electronic Science and Technology, 2025, 38(4): 25-30.
[2]	WANG Yong, YANG Yilong, FAN Xiaohui, ZHOU Lei, KONG Xiangyong. Brain Tumor Classification Algorithm Based on Transfer Learning and Improved EfficientNet-B0 [J]. Electronic Science and Technology, 2025, 38(4): 46-51.
[3]	XU Yaobo, YANG Xinqiang, XU Guangchao, YANG Shihao, DUAN Guoyong. Transformer Fault Identification Method Based on Gramian Angle Difference Field and CNN-BiGRU [J]. Electronic Science and Technology, 2025, 38(4): 73-79.
[4]	ZHAO Yunfei, XUE Cunjin. A Remote Sensing Image Water Extraction Method by Combining Atrous Convolution and Pooling Models [J]. Electronic Science and Technology, 2025, 38(3): 40-46.
[5]	CHEN Yuyang, LI Feng. Integration of CNN and Transformer for Retinal OCT Image Fluid Segmentation Method [J]. Electronic Science and Technology, 2025, 38(3): 47-59.
[6]	ZHENG Fangliang, WANG Yannian, LIAN Jihong, RUAN Pei. Face Image Super-Resolution Reconstruction Based on Conditional Priori Swin Transformer [J]. Electronic Science and Technology, 2025, 38(2): 35-41.
[7]	LAI Ying, JU Zhiyong, YE Yuxin. A Vehicle Detection Algorithm Based on Improved YOLOv4 [J]. Electronic Science and Technology, 2025, 38(1): 81-87.
[8]	KUAI Xinchen, LI Ye. Hybrid Image Super-Resolution Reconstruction with Multiple and Multi-Scale Attention [J]. Electronic Science and Technology, 2024, 37(9): 34-42.
[9]	HE Xing, HUANG Yongming, ZHU Yong. Pavement Pothole Detection Method Based on Improved YOLOv5 [J]. Electronic Science and Technology, 2024, 37(7): 53-59.
[10]	ZHAO Xu, HU Demin. Multi-Path Parallel Multi-Scale Feature Reuse for Remote Sensing Image Super-Resolution [J]. Electronic Science and Technology, 2024, 37(6): 61-68.
[11]	YE Yuxin, JU Zhiyong, LAI Ying. Traffic Sign Detection Algorithm Incorporating Receptive Field Enhancement Module and Attention Mechanism [J]. Electronic Science and Technology, 2024, 37(6): 8-16.
[12]	ZHU Zihao, SONG Yan. Lightweight Capsule Network Fusing Attention and Capsule Pooling [J]. Electronic Science and Technology, 2024, 37(5): 1-8.
[13]	WANG Xiaozhu,YU Lianzhi. Small Object Detection Based on Convolution and Self-Attention of Aggregation [J]. Electronic Science and Technology, 2024, 37(2): 14-22.
[14]	NAN Jiao,SUN Zhanquan. Sorting Method of Multi Leads ECG Based on Mutual Information [J]. Electronic Science and Technology, 2024, 37(2): 55-60.
[15]	CUI Jingnan, HUANG Chunyan, LI Yanling. Improvement of YOLOv5s Algorithm for Steel Surface Defect Detection [J]. Electronic Science and Technology, 2024, 37(12): 48-55.