融合通道全局注意力机制的双路径实时语义分割

doi:10.16180/j.cnki.issn1007-7820.2025.05.012

Abstract

Abstract:

In view of the problems of the mainstream real-time semantic segmentation methods, such as poor multi-scale feature extraction ability, weak feature extraction ability of lightweight backbone network, and lack of effective fusion of context information, a two-path real-time semantic segmentation model with global attention mechanism is proposed in this study. The FPPM(Fast Pyramid Pooling Module) is lightweight based on the PPM(Pyramid Pooling Module) to improve the speed of the module while maintaining multi-scale information extraction. Spatial information branching can compensate for the performance loss caused using lightweight backbone networks. The channel global attention mechanism effectively integrates spatial information and semantic information extracted by backbone network, and interacts with global information to improve the segmentation performance of the model. Without pre-training with other data sets, the proposed model achieves 73.8% average cross ratio on PASCAL VOC2012 validation data set with 14.6 MB of reference, and reaches 43 frame∙s^-1 on NVIDIA TITAN Xp,which indicates that the model achieves a good balance in accuracy and speed.

Key words: semantic segmentation, dual-path, attention mechanism, multi-scale pooling, real-time, deep learning, information fusion, lightweight

CLC Number:

TP391

HU Jinlei, TIAN Engang, QU Feng. Dual-Path Real-Time Semantic Segmentation Network with Channel-Level Global Attention Mechanism[J].Electronic Science and Technology, 2025, 38(5): 83-88.

Figures/Tables 8

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Table 1.

Figure 6.

Table 2.

References 24

[1]	Mo Y, Wu Y, Yang X, et al. Review the state-of-the-art technologies of semantic segmentation based on deep learning[J]. Neurocomputing, 2022, 49(3):626-646.
[2]	于润润, 姜晓燕, 朱凯赢, 等. 基于上下文注意力机制的实时语义分割[J]. 电子科技, 2022, 35(12):57-63.
	Yu Runrun, Jiang Xiaoyan, Zhu Kaiying, et al. Real-time image semantic segmentation based on contextual attention mechanism[J]. Electronic Science and Technology, 2022, 35(12):57-63.
[3]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]. Boston: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015:3431-3440.
[4]	Badrinarayanan V, Kendall A, Cipolla R. Segnet:A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. doi: 10.1109/TPAMI.2016.2644615 pmid: 28060704
[5]	Zhao H S, Shi J P, Qi X J, et al. Pyramid scene parsing network[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:2881-2890.
[6]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:770-778.
[7]	Chen L C, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets,atrous convolution, and fully connected CRFS[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4):834-848.
[8]	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image se-gmentation[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:801-818.
[9]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:7132-7141.
[10]	Fu J, Liu J, Tian H, et al. Dual attention network for scene segmentation[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019:3146-3154.
[11]	Huang Z, Wang X, Huang L, et al. Ccnet:Criss-cross attention for semantic segmentation[C]. Long Beach: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019:603-612.
[12]	Li X, Zhong Z, Wu J, et al. Expectation-maximization attention networks for semantic segmentation[C]. Long Beach: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019:9167-9176.
[13]	Guo M H, Liu Z N, Mu T J, et al. Beyond self-attention: External attention using two linear layers for visual tasks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(5):5436-5447.
[14]	Woo S, Park J, Lee J Y, et al. Cbam:Convolutional block attention module[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:3-19.
[15]	Hou Q, Zhou D, Feng J. Coordinate attention for efficient mobile network design[C]. Paris: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021:13713-13722.
[16]	Maaz M, Shaker A, Cholakkal H, et al. Edgenext:Efficiently amalgamated CNN-transformer architecture for mobile vision applications[C]. Cham: European Conference on Computer Vision, 2022:3-20.
[17]	Zhao H, Qi X, Shen X, et al. ICNet for real-time semantic segmentation on high-resolution images[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:405-420.
[18]	Sandler M, Howard A, Zhu M, et al. MobileNetV2:Inverted residuals and linear bottlenecks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:4510-4520.
[19]	Howard A, Sandler M, Chu G, et al. Searching for MobileNetV3[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019:1314-1324.
[20]	Chollet F. Xception:Deep learning with depthwise separable convolutions[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:1251-1258.
[21]	Yu C, Wang J, Peng C, et al. Bisenet: Bilateral segmentation network for real-time semantic segmentation[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:325-341.
[22]	Fan M, Lai S, Huang J, et al. Rethinking bisenet for real-time semantic segmentation[C]. Paris: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021:9716-9725.
[23]	Peng J, Liu Y, Tang S, et al. PP-LiteSeg:A superior real-time semantic segmentation model[EB/OL].(2022-04-06)[2022-11-12]. https://arxiv.org/pdf/2204.02681.pdf.
[24]	Everingham M, Van Gool L, Williams C K I, et al. The pascal visual object classes challenge[J]. International Journal of Computer Vision, 2010, 88(1):303-338.

模型	主干网络	FPS/ frame∙s^-1	mIoU/%
MobileNetV3	MobileNetV3	-	64.7
BiSeNet	ResNet18	85.0	67.5
BiSeNetV2	STDC	57.0	71.4
PSPNet	MobileNetV3	28.0	71.2
DeeplabV3+	MobileNetV3	1.5	72.8
EANet	MobileNetV3	49.0	71.7
本文模型	MobileNetV3	43.0	73.8

快速金字塔池化模块	空间信息路径	通道全局注意力机制	mIoU/%
-	-	-	64.7
√	-	-	69.2
-	√	-	66.8
√	√	√	73.8

Dual-Path Real-Time Semantic Segmentation Network with Channel-Level Global Attention Mechanism

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 24

Related Articles 15

Metrics

Comments

Recommended 0

[1]	CHEN Manman, YU Lianzhi. Deep BiGRU and DPCS Sentiment Analysis Model Combined with Coattention Network [J]. Electronic Science and Technology, 2025, 38(5): 22-30.
[2]	ZHAO Wenqi, ZHANG Lixin, KAN Xi, ZHENG Haoren. Outdoor Garbage Detection and Recognition Based on Dual Branch Networks [J]. Electronic Science and Technology, 2025, 38(4): 1-9.
[3]	BI Hanjia, YANG Churui, WANG Xiaoyu, HUANG Yuehua. Multi-Class Defect Target Detection for Transmission Lines Based on Improved YOLOv7 [J]. Electronic Science and Technology, 2025, 38(4): 16-24.
[4]	FU Peng, SONG Xiaoxia. Seismic Data Denoising Method Based on CycleGAN [J]. Electronic Science and Technology, 2025, 38(4): 25-30.
[5]	CHEN Li, MA Zhuang, YIN Zhong. DEVAE-GAN:A Generative Model of the fNIRS Data under Multiple Levels of the Cognitive Workload [J]. Electronic Science and Technology, 2025, 38(4): 31-38.
[6]	WANG Yong, YANG Yilong, FAN Xiaohui, ZHOU Lei, KONG Xiangyong. Brain Tumor Classification Algorithm Based on Transfer Learning and Improved EfficientNet-B0 [J]. Electronic Science and Technology, 2025, 38(4): 46-51.
[7]	XU Yaobo, YANG Xinqiang, XU Guangchao, YANG Shihao, DUAN Guoyong. Transformer Fault Identification Method Based on Gramian Angle Difference Field and CNN-BiGRU [J]. Electronic Science and Technology, 2025, 38(4): 73-79.
[8]	ZHOU Kai, YU Lianzhi. Self-Activated Learning Method for Weakly Supervised Semantic Segmentation Integrating Attention Mechanism [J]. Electronic Science and Technology, 2025, 38(4): 80-86.
[9]	WANG Ziyi, CHEN Shiping. Self-Supervised Network Intrusion Detection Model Based on Graph Contrastive Learning [J]. Electronic Science and Technology, 2025, 38(3): 22-31.
[10]	ZHAO Yunfei, XUE Cunjin. A Remote Sensing Image Water Extraction Method by Combining Atrous Convolution and Pooling Models [J]. Electronic Science and Technology, 2025, 38(3): 40-46.
[11]	CHEN Yuyang, LI Feng. Integration of CNN and Transformer for Retinal OCT Image Fluid Segmentation Method [J]. Electronic Science and Technology, 2025, 38(3): 47-59.
[12]	ZHENG Fangliang, WANG Yannian, LIAN Jihong, RUAN Pei. Face Image Super-Resolution Reconstruction Based on Conditional Priori Swin Transformer [J]. Electronic Science and Technology, 2025, 38(2): 35-41.
[13]	MA Zhuang, GAN Kaiyu, YIN Zhong. Emotion Recognition Based on Multimodal Fusion of the EEG and Peripheral Physiological Signals [J]. Electronic Science and Technology, 2025, 38(2): 62-69.
[14]	LAI Ying, JU Zhiyong, YE Yuxin. A Vehicle Detection Algorithm Based on Improved YOLOv4 [J]. Electronic Science and Technology, 2025, 38(1): 81-87.
[15]	KUAI Xinchen, LI Ye. Hybrid Image Super-Resolution Reconstruction with Multiple and Multi-Scale Attention [J]. Electronic Science and Technology, 2024, 37(9): 34-42.