融合通道全局注意力机制的双路径实时语义分割

doi:10.16180/j.cnki.issn1007-7820.2025.05.012

电子科技 ›› 2025, Vol. 38 ›› Issue (5): 83-88.doi: 10.16180/j.cnki.issn1007-7820.2025.05.012

融合通道全局注意力机制的双路径实时语义分割

胡锦磊¹, 田恩刚¹(), 瞿枫²

1.上海理工大学光电信息与计算机工程学院,上海 200093
2.常州市住房公积金管理中心,江苏常州 213003

收稿日期:2023-11-14 修回日期:2023-12-13 出版日期:2025-05-15 发布日期:2025-05-14
通讯作者: 田恩刚 E-mail:tianengang@163.com
作者简介:胡锦磊(1997-),男,硕士研究生。研究方向:计算机视觉。
基金资助:
国家自然科学基金(62173231)

Dual-Path Real-Time Semantic Segmentation Network with Channel-Level Global Attention Mechanism

HU Jinlei¹, TIAN Engang¹(), QU Feng²

1. School of Optical-Electrical and Computer Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China
2. Changzhou Housing Provident Fund Management Center,Changzhou 213003,China

Received:2023-11-14 Revised:2023-12-13 Online:2025-05-15 Published:2025-05-14
Contact: TIAN Engang E-mail:tianengang@163.com
Supported by:
National Natural Science Foundation of China(62173231)

摘要/Abstract

摘要：

针对主流实时语义分割方法存在多尺度特征提取能力较差、轻量型骨干网络特征提取能力较弱、缺少上下文信息的有效融合等问题,文中提出一种融合通道全局注意力机制的双路实时语义分割模型。快速金字塔池化模块(Fast Pyramid Pooling Module, FPPM)在金字塔池化模块(Pyramid Pooling Module, PPM)基础上进行轻量化设计,在保持多尺度信息提取的同时提高模块的速度。空间信息分支可以弥补使用轻量级骨干网络产生的性能损失。通道全局注意力机制对空间信息和骨干网络提取的语义信息进行有效融合,并对全局信息进行交互,提高模型的分割性能。在不使用其他数据集预训练的情况下,所提模型在PASCAL VOC2012验证数据集上的平均交并比为73.8%,参数量为14.6 MB,在NVIDIA TITAN Xp上的每秒帧数可以达到43 frame∙s^-1,表明该模型在精度和速度上实现了较好平衡。

关键词: 语义分割, 双路径, 注意力机制, 多尺度池化, 实时, 深度学习, 信息融合, 轻量化

Abstract:

In view of the problems of the mainstream real-time semantic segmentation methods, such as poor multi-scale feature extraction ability, weak feature extraction ability of lightweight backbone network, and lack of effective fusion of context information, a two-path real-time semantic segmentation model with global attention mechanism is proposed in this study. The FPPM(Fast Pyramid Pooling Module) is lightweight based on the PPM(Pyramid Pooling Module) to improve the speed of the module while maintaining multi-scale information extraction. Spatial information branching can compensate for the performance loss caused using lightweight backbone networks. The channel global attention mechanism effectively integrates spatial information and semantic information extracted by backbone network, and interacts with global information to improve the segmentation performance of the model. Without pre-training with other data sets, the proposed model achieves 73.8% average cross ratio on PASCAL VOC2012 validation data set with 14.6 MB of reference, and reaches 43 frame∙s^-1 on NVIDIA TITAN Xp,which indicates that the model achieves a good balance in accuracy and speed.

Key words: semantic segmentation, dual-path, attention mechanism, multi-scale pooling, real-time, deep learning, information fusion, lightweight

中图分类号:

TP391

胡锦磊, 田恩刚, 瞿枫. 融合通道全局注意力机制的双路径实时语义分割[J]. 电子科技, 2025, 38(5): 83-88.

HU Jinlei, TIAN Engang, QU Feng. Dual-Path Real-Time Semantic Segmentation Network with Channel-Level Global Attention Mechanism[J]. Electronic Science and Technology, 2025, 38(5): 83-88.

图/表 8

图1

图2

图3

图4

图5

表1

图6

表2

参考文献 24

[1]	Mo Y, Wu Y, Yang X, et al. Review the state-of-the-art technologies of semantic segmentation based on deep learning[J]. Neurocomputing, 2022, 49(3):626-646.
[2]	于润润, 姜晓燕, 朱凯赢, 等. 基于上下文注意力机制的实时语义分割[J]. 电子科技, 2022, 35(12):57-63.
	Yu Runrun, Jiang Xiaoyan, Zhu Kaiying, et al. Real-time image semantic segmentation based on contextual attention mechanism[J]. Electronic Science and Technology, 2022, 35(12):57-63.
[3]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]. Boston: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015:3431-3440.
[4]	Badrinarayanan V, Kendall A, Cipolla R. Segnet:A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. doi: 10.1109/TPAMI.2016.2644615 pmid: 28060704
[5]	Zhao H S, Shi J P, Qi X J, et al. Pyramid scene parsing network[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:2881-2890.
[6]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:770-778.
[7]	Chen L C, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets,atrous convolution, and fully connected CRFS[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4):834-848.
[8]	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image se-gmentation[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:801-818.
[9]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:7132-7141.
[10]	Fu J, Liu J, Tian H, et al. Dual attention network for scene segmentation[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019:3146-3154.
[11]	Huang Z, Wang X, Huang L, et al. Ccnet:Criss-cross attention for semantic segmentation[C]. Long Beach: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019:603-612.
[12]	Li X, Zhong Z, Wu J, et al. Expectation-maximization attention networks for semantic segmentation[C]. Long Beach: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019:9167-9176.
[13]	Guo M H, Liu Z N, Mu T J, et al. Beyond self-attention: External attention using two linear layers for visual tasks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(5):5436-5447.
[14]	Woo S, Park J, Lee J Y, et al. Cbam:Convolutional block attention module[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:3-19.
[15]	Hou Q, Zhou D, Feng J. Coordinate attention for efficient mobile network design[C]. Paris: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021:13713-13722.
[16]	Maaz M, Shaker A, Cholakkal H, et al. Edgenext:Efficiently amalgamated CNN-transformer architecture for mobile vision applications[C]. Cham: European Conference on Computer Vision, 2022:3-20.
[17]	Zhao H, Qi X, Shen X, et al. ICNet for real-time semantic segmentation on high-resolution images[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:405-420.
[18]	Sandler M, Howard A, Zhu M, et al. MobileNetV2:Inverted residuals and linear bottlenecks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:4510-4520.
[19]	Howard A, Sandler M, Chu G, et al. Searching for MobileNetV3[C]. Long Beach: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019:1314-1324.
[20]	Chollet F. Xception:Deep learning with depthwise separable convolutions[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:1251-1258.
[21]	Yu C, Wang J, Peng C, et al. Bisenet: Bilateral segmentation network for real-time semantic segmentation[C]. Munich: Proceedings of the European Conference on Computer Vision, 2018:325-341.
[22]	Fan M, Lai S, Huang J, et al. Rethinking bisenet for real-time semantic segmentation[C]. Paris: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021:9716-9725.
[23]	Peng J, Liu Y, Tang S, et al. PP-LiteSeg:A superior real-time semantic segmentation model[EB/OL].(2022-04-06)[2022-11-12]. https://arxiv.org/pdf/2204.02681.pdf.
[24]	Everingham M, Van Gool L, Williams C K I, et al. The pascal visual object classes challenge[J]. International Journal of Computer Vision, 2010, 88(1):303-338.

模型	主干网络	FPS/ frame∙s^-1	mIoU/%
MobileNetV3	MobileNetV3	-	64.7
BiSeNet	ResNet18	85.0	67.5
BiSeNetV2	STDC	57.0	71.4
PSPNet	MobileNetV3	28.0	71.2
DeeplabV3+	MobileNetV3	1.5	72.8
EANet	MobileNetV3	49.0	71.7
本文模型	MobileNetV3	43.0	73.8

快速金字塔池化模块	空间信息路径	通道全局注意力机制	mIoU/%
-	-	-	64.7
√	-	-	69.2
-	√	-	66.8
√	√	√	73.8

融合通道全局注意力机制的双路径实时语义分割

Dual-Path Real-Time Semantic Segmentation Network with Channel-Level Global Attention Mechanism

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 24

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	陈漫漫, 于莲芝. 结合共注意网络的深度BiGRU和DPCS情感分析模型[J]. 电子科技, 2025, 38(5): 22-30.
[2]	赵文琪, 张立新, 阚希, 郑好韧. 基于双分支网络的户外垃圾检测识别[J]. 电子科技, 2025, 38(4): 1-9.
[3]	毕含嘉, 杨楚睿, 王小雨, 黄悦华. 基于改进YOLOv7的输电线路多类缺陷目标检测[J]. 电子科技, 2025, 38(4): 16-24.
[4]	傅鹏, 宋晓霞. 基于CycleGAN的地震数据去噪方法[J]. 电子科技, 2025, 38(4): 25-30.
[5]	陈利, 马壮, 尹钟. DEVAE-GAN:多级认知工作负荷水平fNIRS数据生成模型[J]. 电子科技, 2025, 38(4): 31-38.
[6]	王勇, 杨义龙, 范晓晖, 周雷, 孔祥勇. 基于迁移学习和改进EfficientNet-B0的脑肿瘤分类算法[J]. 电子科技, 2025, 38(4): 46-51.
[7]	许耀博, 杨信强, 徐广超, 杨诗豪, 段国勇. 基于格拉姆角差场和CNN-BiGRU的变压器故障识别法[J]. 电子科技, 2025, 38(4): 73-79.
[8]	周凯, 于莲芝. 融合注意力机制的弱监督语义分割自激活方法[J]. 电子科技, 2025, 38(4): 80-86.
[9]	王紫祎, 陈世平. 基于图对比学习的自监督网络流量检测模型[J]. 电子科技, 2025, 38(3): 22-31.
[10]	赵云飞, 薛存金. 一种融合空洞卷积与池化模型的遥感影像水体提取方法[J]. 电子科技, 2025, 38(3): 40-46.
[11]	陈宇洋, 李峰. 融合CNN与Transformer的视网膜OCT图像积液分割方法[J]. 电子科技, 2025, 38(3): 47-59.
[12]	郑方亮, 王延年, 廉继红, 阮佩. 基于条件先验Swin Transformer的人脸图像超分辨重建[J]. 电子科技, 2025, 38(2): 35-41.
[13]	马壮, 甘开宇, 尹钟. 基于脑电信号和周围生理信号的多模态融合情感识别[J]. 电子科技, 2025, 38(2): 62-69.
[14]	赖颖, 巨志勇, 叶雨新. 基于改进YOLOv4的车辆检测算法[J]. 电子科技, 2025, 38(1): 81-87.
[15]	蒯新晨, 李烨. 多种及多尺度注意力混合的图像超分辨率重建[J]. 电子科技, 2024, 37(9): 34-42.