基于特征校准的双注意力遮挡行人检测器

doi:10.19665/j.issn1001-2400.20240909

Abstract

Abstract:

One of the major challenges faced by pedestrian detection technology based on computer vision is the issue of occlusion,including inter-class occlusion caused by objects in the natural environment and intra-class occlusion between pedestrians.These intertwined occlusion patterns limit the performance of pedestrian detectors.To address this problem,this paper proposes a dual-attention detection network based on feature calibration within the standard Faster R-CNN pedestrian detection framework.The network first generates attention masks through supervised learning to represent the spatial features of pedestrians in the image.These masks are then fused with backbone features and combined with a channel attention mechanism to calibrate pedestrian regions.This approach enhances the visibility of pedestrian regions while reducing the impact of occluded parts on classification and regression.Additionally,a non-uniform sampling strategy based on occlusion rates is introduced,targeting hard examples to allow the network to better learn complex occlusion patterns.Experimental results demonstrate that in comparison with standard pedestrian detectors,the proposed method achieves a 2.5% performance improvement on the reasonable occlusion subset of the CityPersons validation dataset.

Key words: convolutional neural network, pedestrian detection, dual attention mechanism, feature calibration, hard example mining, occlusion rate

CLC Number:

TP391

TANG Shuyuan, ZHOU Yiqing, LI Jintao, LIU Chang, SHI Jinglin. Dual attention pedestrian detector for occlusion scenario based on feature calibration[J].Journal of Xidian University, 2024, 51(6): 25-39.

Figures/Tables 14

References 47

[1]	LIN C, LU J, WANG G, et al. Graininess-Aware Deep Feature Learning for Robust Pedestrian Detection[J]. IEEE Transactions Image Processing, 2020, 29:3820-3834.
[2]	LIU W, LIAO S, HU W. Efficient Single-Stage Pedestrian Detector by Asymptotic Localization Fitting and Multi-Scale Context Encoding[J]. IEEE Transactions Image Processing, 2020, 29:1413-1425.
[3]	CAO J, PANG Y, HAN J, et al. Taking a Look at Small-Scale Pedestrians and Occluded Pedestrians[J]. IEEE Transactions Image Processing, 2020, 29:3143-3152.
[4]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition(CVPR2016). Piscataway:IEEE, 2016:770-778.
[5]	REN S, HE K, GIRSHICK R, et al. Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(6):1137-1149.
[6]	王子晔, 苗夺谦, 赵才荣. 基于多粒度特征的行人跟踪检测结合算法[J]. 计算机研究与发展, 2020, 57(5):996-1002.
	WANG Zhiye, MIAO Duoqian, ZHAO Cairong, et al. A Pedestrian Tracking Algorithm Based on Multi-Granularity Feature[J]. Journal of Computer Research and Development, 2015, 57(5):996-1002.
[7]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal Loss for Dense Object Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2):318-32.
[8]	REN J, CHEN X, LIU J, et al. Accurate Single Stage Detector Using Recurrent Rolling Convolution[C]// Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition(CVPR2017). Piscataway:IEEE, 2017:5420-5428.
[9]	SHAO Z, CHENG G, MA J, et al. Real-Time and Accurate UAV Pedestrian Detection for Social Distancing Monitoring in COVID-19 Pandemic[J]. IEEE Transactions on Multimedia, 2021, 24:2069-2083.
[10]	XIE J, PANG Y, KHAN M H, et al. Mask-Guided Attention Network and Occlusion-Sensitive Hard Example Mining for Occluded Pedestrian Detection[J]. IEEE Transactions on Image Processing, 2020, 30:3872-3884.
[11]	ZOU F, LI X, XU Q, et al. Correlation-and-Correction Fusion Attention Network for Occluded Pedestrian Detection[J]. IEEE Sensors Journal, 2023, 23(6):6061-6073.
[12]	XIE J, PANG Y, CHOLAKKAL H, et al. PSC-Net:Learning Part Spatial Co-ccurrence for Occluded Pedestrian Detection[J]. Science China Information Sciences, 2021, 64(2):1-13.
[13]	WU J, ZHOU C, ZHANG Q, et al. Self-Mimic Learning for Small-Scale Pedestrian Detection[C]// Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020:2012-2020.
[14]	WANG H, LI Y, WANG S. Fast Pedestrian Detection with Attention-Enhanced Multi-Scale RPN and Soft-Cascaded Decision Trees[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(12):5086-5093.
[15]	ZHANG T, YE Q, ZHANG B, et al. Feature Calibration Network for Occluded Pedestrian Detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 23(5):4151-4163.
[16]	XIE J, CHOLAKKAL H, ANWER R M, et al. Count-andSimilarity-Aware R-CNN for Pedestrian Detection[C]// Proceedings of the 16th European Conference on Computer Vision. Berlin:Springer, 2020:1-16.
[17]	WU J, ZHOU C, YANG M, et al. Temporal Context Enhanced Detection of Heavily Occluded Pedestrians[C]// Proceedings of the 33rd IEEE Conference on Computer Vision and Pattern Recognition(CVPR2020). Piscataway:IEEE, 2020:13430-13439.
[18]	NOH J, LEE S, KIM B, et al. Improving Occlusion and Hard Negative Handling for Single-Stage Pedestrian Detectors[C]// Proceedings of the 31st IEEE Conference on Computer Vision and Pattern Recognition(CVPR2018). Piscataway:IEEE, 2018:966-974.
[19]	ZHOU C, YUAN J. Multi-Label Learning of Part Detectors for Heavily Occluded Pedestrian Detection[C]// Proceedings of the 16th IEEE International Conference on Computer Vision(ICCV2017). Piscataway:IEEE, 2017:3486-3495.
[20]	ZHANG T, HAN Z, XU H, et al. CircleNet:Reciprocating Feature Adaptation for Robust Pedestrian Detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 21(11):4593-4604.
[21]	ZHANG S, YANG J, SCHIELE B. Occluded Pedestrian Detection Through Guided Attention in CNNs[C]// Proceedings of the 31st IEEE Conference on Computer Vision and Pattern Recognition(CVPR2018). Piscataway:IEEE, 2018:6995-7003.
[22]	ZHOU C, YUAN J. Bi-Box Regression for Pedestrian Detection and Occlusion Estimation[C]// Proceedings of the 15th European Conference on Computer Vision(ECCV2018). Berlin:Springer, 2018:135-151.
[23]	汪昱东, 郭继昌, 王天保. 一种改进的雾天图像行人和车辆检测算法[J]. 西安电子科技大学学报, 2020, 47(4):70-77.
	WANG Yudong, GUO Jichang, WANG Tianbao. Algorithm for Foggy-Image Pedestrian and Vehicle Detection[J]. Journal of Xidian University, 2020, 47(4):70-77.
[24]	LI W, HAN S, LIU Y. A Pedestrian Detection Algorithm Based on Channel Attention Mechanism[C]// Proceedings of the 33rd Chinese Control and Decision Conference(CCDC2021). Piscataway:IEEE, 2021:5954-5959.
[25]	LI Y, WANG Q, LIU R. Research on YOLOv3 Pedestrian Detection Algorithm Based on Channel Attention Mechanism[C]// Proceedings of the 1st IEEE International Conference on Computer Science,Electronic Information Engineering and Intelligent Control Technology(CEI2021). Piscataway:IEEE, 2021:229-232.
[26]	WANG X, HU H, ZHANG Y. Pedestrian Detection Based on Spatial Attention Module for Outdoor Video Surveillance[C]// Proceedings of IEEE 5th International Conference on Multimedia Big Data(BigMM2019). Piscataway:IEEE, 2019:247-251.
[27]	FU J, LIU J, TIAN H, et al. Dual Attention Network for Scene Segmentation[C]// Proceedings of the 32nd IEEE Conference on Computer Vision and Pattern Recognition(CVPR2019). Piscataway:IEEE 2019:3141-3149.
[28]	SHRIVASTAVA A, GUPTA A, GIRSHICK R. Training Region-Based Object Detectors with Online Hard Example Mining[C]// Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition(CVPR2016). Piscataway:IEEE, 2016:761-769.
[29]	ZHANG S, BENENSON R, SCHIELE B. Citypersons:A Diverse Dataset for Pedestrian Detection[C]// Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition(CVPR2017). Piscataway:IEEE, 2017:3213-3221.
[30]	DOLLAR P, WOJEK C, SCHIELE B,etal. Pedestrian Detection:An Evaluation of the State of the Art[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 34(4):743-761.
[31]	GAO F, CAI Y, DENG F, et al. Feature Alignment in Anchor-Free Object Detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(8):3799-3810.
[32]	CHEN Y, YE J, WAN X. TF-YOLO:A Transformer-Fusion-Based YOLO Detector for Multimodal Pedestrian Detection in Autonomous Driving Scenes.[J]. World Electric Vehicle Journal. 2023; 14(12):352.
[33]	ZHANG S, WEN L, BIAN X, et al. Occlusion-Aware R-CNN:Detecting Pedestrians in a Crowd[C]// Proceedings of the 15th European Conference on Computer Vision(ECCV2018). Berlin:Springer, 2018:637-653.
[34]	TIAN Y, LUO P, WANG X, et al. DeepLearning Strong Parts for Pedestrian Detection[C]// Proceedings of the 15th IEEE International Conference on Computer Vision(ICCV). Piscataway:IEEE, 2015:1904-1912.
[35]	QIU T, WANG L, WANG P, et al. Research on Object Detection Algorithm Based on Improved YOLOv5[J]. Journal of Computer Engineering & Applications, 2022, 58(13):63-73.
[36]	CONG C, YANG Z, SONG Y, et al. Towards Enforcing Social Distancing Regulations with Occlusion-Aware Crowd Detection[C]// Proceedings of the 16th International Conference on Control,Automation,Robotics and Vision(ICARCV2020). Piscataway:IEEE, 2020:297-302.
[37]	SHAO S, ZHAO Z, LI B, et al. Crowdhuman:A Benchmark for Detecting Human in a Crowd(2023)[J/OL]. [2023-6-10].https://doi.org/10.48550/arXiv.1805.00123.
[38]	CHEN K, WANG J, PANG J, et al. MMDetection:OpenMmlab Detection Toolbox and Benchmark(2023)[J/OL]. [2023-6-17].https://doi.org/10.48550/arXiv.1906.07155.
[39]	KINGMA D P, BA J. Adam:A Method for Stochastic Optimization(2023)[J/OL]. [2023-6-22].https://doi.org/10.48550/arXiv.1412.6980.
[40]	LIN Z, PEI W, CHEN F, et al. Pedestrian Detection by Exemplar-Guided Contrastive Learning[J]. IEEE Transactions on Image Processing, 2022, 32:2003-2016
[41]	WANG X, XIAO T, JIANG Y, et al. Repulsion Loss:Detecting Pedestrians in a Crowd[C]// Proceedings of the 31st IEEE Conference on Computer Vision and Pattern Recognition(CVPR2018). Piscataway:IEEE, 2018:7774-7783.
[42]	ZHANG S, BENENSON R, OMRAN M, et al. How Far Are We From Solving Pedestrian Detection?[C]// Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition(CVPR2016). Piscataway:IEEE, 2016:1259-1267.
[43]	BRAZIL G, LIU X. Pedestrian Detection with Autoregressive Network Phases[C]// Proceedings of the 32nd IEEE Conference on Computer Vision and Pattern Recognition(CVPR2019). Piscataway:IEEE, 2019:7231-7240.
[44]	BRAZIL G, YIN X, LIU X. Illuminating Pedestrians via Simultaneous Detection & Segmentation[J]. IEEE International Conference on Computer Vision, 2017:4950-4959.
[45]	LIU W, ANGUELOV D, ERHAN D, et al. SSD:Single Shot Multibox Detector[C]// Proceedings of Computer Vision-ECCV 2016:14th European Conference. Berlin:Springer, 2016:21-37.
[46]	LIN T Y, GOYAL P, GIRSHICK R, et al. Focal Loss for Dense Object Detection(2023)[J/OL]. [2023-6-27].https://doi.org/10.48550/arXiv.1708.02002.
[47]	REN S, HE K, GIRSHICK R B, et al. Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39:1137-1149.

类别	特点概述	缺点	代表性方法
基于子部件模型的方法	基于子部件模型的方法通过分割和独立处理行人的各个部件,以及综合各部件的检测结果,适用于需要高精度和鲁棒性的应用场景。	复杂性高,需要大量计算资源,并且设计正确的部件分割可能具有挑战性,限制了模型在复杂场景下的适应能力。	CircleNet^[20], OR-CNN^[33]
基于注意力机制的方法	基于注意力机制的行人检测模型通过引入注意力机制来增强和校准行人特征,从而提升检测器在遮挡情况下的准确性和鲁棒性。	注意力分配可能不准确,增加了计算成本,而且对模型结构的依赖性较强,设计和优化相对复杂。	MGAN^[10], YOLOv3^[25], TF-YOLO^[32], YOLO-G^[35]

方法	R	Δ	HO	Δ	R+HO	Δ
Baseline	13.2		46.8		29.5
Baseline+SSA	12.5	0.7	43.6	3.2	26.2	3.3
Baseline+SSA+PrC(DAFC)	11.3	1.9	40.6	6.2	23.4	6.1
DAFC+NUHEM	10.7	2.5	39.6	7.2	22.6	6.9

方法	Scale	R/%	HO/%
ATT-self	×1	20.9	58.3
ATT-vbb	×1	16.4	57.3
ATT-part	×1	16.0	56.7
EGCL	×1	11.5	51.1
DAFC	×1	11.2	46.7

方法	R	HO
基线网络	11.9	45.6
OHEM	11.0	43.2
Focal loss	11.2	44.3
NUHEM	11.8	44.8

方法	Scale	R/%	HO/%
ATT-part	×1	16.0	56.7
CCFA-Net	×1	15.4	52.9
RepLoss	×1	13.2	56.9
ALFNet	×1	12.0	51.9
OR-CNN	×1	12.8	55.7
FC-Net	×1	13.5	44.3
EGCL	×1	10.9	46.4
DAFC+	×1	11.8	44.8
CCFA-Net	×1.3	13.4	50.0
RepLoss	×1.3	11.6	55.3
OR-CNN	×1.3	11.0	51.3
CircleNet	×1.3	11.8	50.2
FC-Net	×1.3	11.6	42.8
EGCL	×1.3	10.5	45.3
DAFC+	×1.3	10.7	39.6

Dual attention pedestrian detector for occlusion scenario based on feature calibration

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 47

Related Articles 15

Metrics

Comments

Recommended 0

方法	R	HO	R+HO
DeepParts	11.9	60.0	22.8
ATT-vbb	10.3	45.2	18.2
Adaptive FasterRCNN	9.2	57.6	20.0
Bi-box	7.6	44.4	16.1
SDS-RCNN	7.4	58.6	19.7
AR-Ped	6.5	48.8	16.1
DAFC+	6.9	37.1	12.9

方法	mAP50	mAP50∶95
SSD	69.6	35.5
RetinaNet	76.1	45.7
Faster-RCNN	79.3	46.6
YOLOv5-l	80.2	50.4
YOLOX-l	81.0	49.0
YOLOv7	83.1	52.3
YOLOv8-l	86.3	56.5
DAFC+	89.8	57.2

[1]	QU Jiahui, HE Jie, DONG Wenqian, LI Yunsong, ZHANG Tongzhen, YANG Yufei. Change detection method based on multi-scale and multi-resolution information fusion [J]. Journal of Xidian University, 2025, 52(1): 105-116.
[2]	HENG Hongjun, YU Longwei. Time series anomaly detection based on multi-scale feature information fusion [J]. Journal of Xidian University, 2024, 51(3): 203-214.
[3]	GONG Junyang, FU Weihong, FANG Houzhang. Research on lightweight and feature enhancement of SAR image ship targets detection [J]. Journal of Xidian University, 2024, 51(2): 96-106.
[4]	ZHONG Hao, BIAN Shan, WANG Chuntao. Real world image tampering localization combining the self-attention mechanism and convolutional neural networks [J]. Journal of Xidian University, 2024, 51(1): 135-146.
[5]	ZHANG Xinyu, LIANG Yu, ZHANG Wei. Real-time smoke segmentation algorithm combining global and local information [J]. Journal of Xidian University, 2024, 51(1): 147-156.
[6]	HUANG Heyuan, MU Caihong, FANG Yunfei, LIU Yi. Graph convolution neural network for recommendation using graph negative sampling [J]. Journal of Xidian University, 2024, 51(1): 86-99.
[7]	DU Mingyang, DU Meng, PAN Jifei, BI Daping. Generative adversarial model for radar intra-pulse signal denoising and recognition [J]. Journal of Xidian University, 2023, 50(6): 133-147.
[8]	FAN Wentong,LI Zhenyu,ZHANG Tao,LUO Xiangyang. JPEG image steganalysis based on deep extraction of stego noise [J]. Journal of Xidian University, 2023, 50(4): 157-169.
[9]	WANG Kan, WANG Mengyang, LIU Xin, TIAN Guoqiang, LI Chuan, LIU Wei. Event detection by combining self-attention and CNN-BiGRU [J]. Journal of Xidian University, 2022, 49(5): 181-188.
[10]	SHI Yunlong,YUAN Wenhao,HU Shaodong,LOU Yingxi. Convolutional quasi-recurrent network for real-time speech enhancement [J]. Journal of Xidian University, 2022, 49(3): 183-190.
[11]	ZHOU Peng,YANG Jun. Index edge geometric convolution neural network for point cloud classification [J]. Journal of Xidian University, 2022, 49(2): 207-217.
[12]	YU Haoyang,YIN Liang,LI Shufang,LV Shun. Recognition algorithm for the little sample radar modulation signal based on the generative adversarial network [J]. Journal of Xidian University, 2021, 48(6): 96-104.
[13]	CHEN Changchuan,WANG Haining,HUANG Lian,HUANG Tao,LI Lianjie,HUANG Xiangkang,DAI Shaosheng. Facial expression recognition based on local representation [J]. Journal of Xidian University, 2021, 48(5): 100-109.
[14]	SONG Jianfeng,MIAO Qiguang,WANG Chongxiao,XU Hao,YANG Jin. Multi-scale single object tracking based on the attention mechanism [J]. Journal of Xidian University, 2021, 48(5): 110-116.
[15]	ZHANG Yuhao,CHENG Peitao,ZHANG Shuhao,WANG Xiumei. Lightweight image super-resolution with the adaptive weight learning network [J]. Journal of Xidian University, 2021, 48(5): 15-22.