基于深度学习的行为识别方法

doi:10.16180/j.cnki.issn1007-7820.2024.10.009

Abstract

Abstract:

The key of current research on behavior recognition algorithms based on deep learning lies in enhancing the accuracy and stability of key point extraction, in order to achieve more accurate action recognition of targets. However, many current algorithms tend to just add attention mechanisms that appear to perform better in the feature extraction stage of the target, without considering the impact of different attention mechanisms on different models and tasks. Therefore, this study proposes an algorithmic model for pose estimation based on various attention mechanisms, which further highlights the importance of selecting an appropriate attention mechanism by comparing the impact of different attention mechanisms on the model. In addition, considering the stability of key point extraction, the initialization of the model is fine-tuned to select a more suitable initialization method that improves the performance by increasing the category of weights on network layer judgments. Compared with the performance of the benchmark network model, the model enhances all evaluation metrics on both multiscale and no-multiscale CrowdPose datasets, where the average accuracy improvement in both cases is more than 1%.

Key words: behavior recognition, pose estimation, computer vision, graph convolution network, key points, HRNet, attention mechanism, average precision

CLC Number:

TP391

XIN Tenghao, LI Feifei. Research on Action Recognition Method Based on Deep Learning[J].Electronic Science and Technology, 2024, 37(10): 64-70.

Figures/Tables 7

Figure 1.

Figure 2.

Figure 3.

Table 1.

Table 2.

Table 3.

Table 4.

References 22

[1]	Mitchell T M. Machine learning[M]. New York: McGraw-hill,2007:1-282.
[2]	Le Cun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[3]	Lin W, Sun M T, Poovandran R, et al. Human activity recog-nition for video surveillance[C]. Seattle: IEEE International Symposium on Circuits and Systems,2008:2737-2740.
[4]	Simonyan K, Zisserman A. Two-stream convolutional net-works for action recognition in videos[J]. Advances in Neural Information Processing Systems, 2014, 27(6):568-576.
[5]	Zong M, Wang R, Chen X, et al. Motion saliency based multi-stream multiplier ResNets for action recognition[J]. Image and Vision Computing, 2021, 10(7):104-108.
[6]	Tran D, Bourdev L, Fergus R, et al. Learning spatiotemporal features with 3D convolutional networks[C]. Santiago: Proceedings of the IEEE International Conference on Computer Vision,2015:4489-4497.
[7]	Du W, Wang Y, Qiao Y. Rpan:An end-to-end recurrent pose-attention network for action recognition in videos[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision,2017:3725-3734.
[8]	Sun L, Jia K, Chen K, et al. Lattice long short-term memory for human action recognition[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision,2017:2147-2156.
[9]	Perrett T, Damen D. DDLSTM:Dual-domain LSTM for cross-dataset action recognition[C]. Los Angeles: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:7852-7861.
[10]	Meng Y, Lin C C, Panda R, et al. Arnet:Adaptive frame resolution for efficient action recognition[C]. Glasgow:Computer Vision-ECCV: The Sixteenth European Conference,2020: 86-104.
[11]	Li M, Chen S, Chen X, et al. Actional-structural graph convolutional networks for skeleton-based action recognition[C]. Los Angeles: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:3595-3603.
[12]	Cheng K, Zhang Y, He X, et al. Skeleton-based action reco-gnition with shift graph convolutional network[C]. Seattle: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:183-192.
[13]	Song Y F, Zhang Z, Shan C, et al. Constructing stronger and faster baselines for skeleton-based action recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(2):1474-1488.
[14]	苏波, 柴自强, 王莉, 等. 基于姿态估计的八段锦序列动作识别与评估[J]. 电子科技, 2022, 35(12):84-90.
	Su Bo, Chai Ziqiang, Wang Li, et al. Eight-section brocade sequence action recognition and evaluation based on poseestimation[J]. Electronic Science and Technology, 2022, 35(12):84-90.
[15]	Newell A, Yang K, Deng J. Stacked hourglass networks for human pose estimation[C]. Amsterdam:Computer Vision-ECCV: The Fourteenth European Conference,2016:483-499.
[16]	Toshev A, Szegedy C. Deeppose:Human pose estimation via deep neural networks[C]. Columbus: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2014:1653-1660.
[17]	Wei S E, Ramakrishna V, Kanade T, et al. Convolutional pose machines[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016: 4724-4732.
[18]	Cheng B, Xiao B, Wang J, et al. Higherhrnet: Scaleaware representation learning for bottomup human pose estimation[C]. Seattle: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:5386-5395.
[19]	Geng Z, Sun K, Xiao B, et al. Bottomup human pose estimation via disentangled keypoint regression[C]. Online: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:14676-14686.
[20]	Sun K, Xiao B, Liu D, et al. Deep high-resolution representation learning for human pose estimation[C]. Los Angeles: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:5693-5703.
[21]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Salt Lake City: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2018:7132-7141.
[22]	Luo Z, Wang Z, Huang Y, et al. Rethinking the heatmap regression for bottomup human pose estimation[C]. Seattle: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:13264-13273.

评价指标	DEKR+Transformer	DEKR+CBAM	HrHRNet(W48)^[22]	HrHRNet+Adaptive^[22]	基准网络^[19]	本文方法
AP	0.670	0.677	0.676	0.675	0.670	0.683
AP50	0.846	0.856	0.874	0.861	0.854	0.858
AP75	0.726	0.734	0.726	0.726	0.724	0.738
AP(E)	0.751	0.755	0.758	0.755	0.755	0.764
AP(M)	0.680	0.687	0.681	0.682	0.680	0.693
AP(H)	0.568	0.578	0.589	0.582	0.569	0.583

评价指标	DEKR+Transformer	DEKR+CBAM	HrHRNet(W48)^[22]	HrHRNet+Adaptive^[22]	基准网络^[19]	本文方法
AP	0.654	0.658	0.659	0.649	0.657	0.668
AP50	0.855	0.861	0.864	0.845	0.857	0.866
AP75	0.709	0.709	0.706	0.696	0.704	0.720
AP(E)	0.721	0.723	0.733	0.727	0.730	0.739
AP(M)	0.664	0.667	0.665	0.655	0.664	0.676
AP(H)	0.566	0.575	0.579	0.561	0.575	0.586

评价指标	基准网络	基准网络+SE	基准网络+weight	本文方法
AP	0.670	0.677	0.676	0.683
AP50	0.854	0.855	0.856	0.858
AP75	0.724	0.731	0.731	0.738
AP(E)	0.755	0.763	0.756	0.764
AP(M)	0.680	0.686	0.685	0.693
AP(H)	0.569	0.575	0.576	0.583

评价指标	基准网络	基准网络+SE	基准网络+weight	本文方法
AP	0.657	0.666	0.658	0.668
AP50	0.857	0.862	0.856	0.866
AP75	0.704	0.715	0.706	0.720
AP(E)	0.730	0.739	0.730	0.739
AP(M)	0.664	0.673	0.666	0.676
AP(H)	0.575	0.581	0.579	0.586

Research on Action Recognition Method Based on Deep Learning

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 22

Related Articles 15

Metrics

Comments

Recommended 0

[1]	KUAI Xinchen, LI Ye. Hybrid Image Super-Resolution Reconstruction with Multiple and Multi-Scale Attention [J]. Electronic Science and Technology, 2024, 37(9): 34-42.
[2]	HE Xing, HUANG Yongming, ZHU Yong. Pavement Pothole Detection Method Based on Improved YOLOv5 [J]. Electronic Science and Technology, 2024, 37(7): 53-59.
[3]	YE Yuxin, JU Zhiyong, LAI Ying. Traffic Sign Detection Algorithm Incorporating Receptive Field Enhancement Module and Attention Mechanism [J]. Electronic Science and Technology, 2024, 37(6): 8-16.
[4]	ZHU Zihao, SONG Yan. Lightweight Capsule Network Fusing Attention and Capsule Pooling [J]. Electronic Science and Technology, 2024, 37(5): 1-8.
[5]	FENG Ting, YING Jie, YANG Haima, LI Fang. Detection of Cervical Lesions Based on Multi-Scale Features and Attention Mechanism [J]. Electronic Science and Technology, 2024, 37(10): 30-39.
[6]	ZHANG Hui, LI Feifei. Emotion Recognition Algorithm Based on Multimodal Cross-Interaction [J]. Electronic Science and Technology, 2024, 37(10): 81-87.
[7]	WANG Jiawei,YU Xiao. Review of Text Classification Research Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(1): 81-86.
[8]	SUN Xi,YU Lianzhi. Image Dehazing Algorithm Based on Residual Attention and Semi-Supervised Learning [J]. Electronic Science and Technology, 2023, 36(9): 50-57.
[9]	DANG Xiaofang,CAI Xingyu. Transformer-Based Maneuvering Target Tracking [J]. Electronic Science and Technology, 2023, 36(9): 86-92.
[10]	SUN Hong,ZHAO Yingzhi. Lightweight Generative Adversarial Networks Based on Multi-Scale Gradient [J]. Electronic Science and Technology, 2023, 36(7): 32-38.
[11]	ZHA Junwei,ZHANG Hongyan. Dynamic Receptive Field Feature Selection Dehazing Network [J]. Electronic Science and Technology, 2023, 36(7): 56-63.
[12]	LIU Yuwei,CAO Min,FENG Haojia. CNAS Recognition Criteria Automatic Benchmarking System Based on Natural Language Processing [J]. Electronic Science and Technology, 2023, 36(5): 28-33.
[13]	SHI Jianke,QIAO Meiying,LI Bingfeng,ZHAO Yan. Underwater Occlusion Target Detection Algorithm Based on Attention Mechanism [J]. Electronic Science and Technology, 2023, 36(5): 62-70.
[14]	SUN Hong,ZHANG Yuxiang. Super-Resolution Image Reconstruction Algorithm Based on Multi-Feature Gated Feedback Residual Network [J]. Electronic Science and Technology, 2023, 36(4): 65-70.
[15]	SUN Hong,LU Meike. Hybrid Recommendation Algorithm Fused with User Behavior Sequence Prediction [J]. Electronic Science and Technology, 2023, 36(4): 84-89.