基于深度学习的行为识别方法

doi:10.16180/j.cnki.issn1007-7820.2024.10.009

摘要/Abstract

摘要：

基于深度学习行为识别算法的关键在于提高关键点提取的准确性和稳定性,以此来更准确地识别目标动作。然而,较多算法在目标特征提取阶段仅加入看似具有较好功能的注意力机制,忽略了不同注意力机制对不同模型和任务的影响。因此,文中提出了一种基于不同注意力机制的姿态估计算法模型。该方法通过比较不同注意力机制对模型的影响,进一步说明了选择注意力机制的重要性。同时,考虑到关键点提取的稳定性,对模型的初始化进行微调,通过增加权重判断网络层类别,选择更合适的初始化方法以提高性能。相较于基准网络模型,该模型在多尺度和无多尺度CrowdPose数据集上所有的评价指标均有所提升。其中,平均精度在两种情况下的提升均超过了1%。

关键词: 行为识别, 姿态估计, 计算机视觉, 图卷积神经网络, 关键点, HRNet, 注意力机制, 平均精度

Abstract:

The key of current research on behavior recognition algorithms based on deep learning lies in enhancing the accuracy and stability of key point extraction, in order to achieve more accurate action recognition of targets. However, many current algorithms tend to just add attention mechanisms that appear to perform better in the feature extraction stage of the target, without considering the impact of different attention mechanisms on different models and tasks. Therefore, this study proposes an algorithmic model for pose estimation based on various attention mechanisms, which further highlights the importance of selecting an appropriate attention mechanism by comparing the impact of different attention mechanisms on the model. In addition, considering the stability of key point extraction, the initialization of the model is fine-tuned to select a more suitable initialization method that improves the performance by increasing the category of weights on network layer judgments. Compared with the performance of the benchmark network model, the model enhances all evaluation metrics on both multiscale and no-multiscale CrowdPose datasets, where the average accuracy improvement in both cases is more than 1%.

Key words: behavior recognition, pose estimation, computer vision, graph convolution network, key points, HRNet, attention mechanism, average precision

中图分类号:

TP391

忻腾浩, 李菲菲. 基于深度学习的行为识别方法[J]. 电子科技, 2024, 37(10): 64-70.

XIN Tenghao, LI Feifei. Research on Action Recognition Method Based on Deep Learning[J]. Electronic Science and Technology, 2024, 37(10): 64-70.

图/表 7

图1

图2

图3

表1

表2

表3

表4

参考文献 22

[1]	Mitchell T M. Machine learning[M]. New York: McGraw-hill,2007:1-282.
[2]	Le Cun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[3]	Lin W, Sun M T, Poovandran R, et al. Human activity recog-nition for video surveillance[C]. Seattle: IEEE International Symposium on Circuits and Systems,2008:2737-2740.
[4]	Simonyan K, Zisserman A. Two-stream convolutional net-works for action recognition in videos[J]. Advances in Neural Information Processing Systems, 2014, 27(6):568-576.
[5]	Zong M, Wang R, Chen X, et al. Motion saliency based multi-stream multiplier ResNets for action recognition[J]. Image and Vision Computing, 2021, 10(7):104-108.
[6]	Tran D, Bourdev L, Fergus R, et al. Learning spatiotemporal features with 3D convolutional networks[C]. Santiago: Proceedings of the IEEE International Conference on Computer Vision,2015:4489-4497.
[7]	Du W, Wang Y, Qiao Y. Rpan:An end-to-end recurrent pose-attention network for action recognition in videos[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision,2017:3725-3734.
[8]	Sun L, Jia K, Chen K, et al. Lattice long short-term memory for human action recognition[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision,2017:2147-2156.
[9]	Perrett T, Damen D. DDLSTM:Dual-domain LSTM for cross-dataset action recognition[C]. Los Angeles: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:7852-7861.
[10]	Meng Y, Lin C C, Panda R, et al. Arnet:Adaptive frame resolution for efficient action recognition[C]. Glasgow:Computer Vision-ECCV: The Sixteenth European Conference,2020: 86-104.
[11]	Li M, Chen S, Chen X, et al. Actional-structural graph convolutional networks for skeleton-based action recognition[C]. Los Angeles: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:3595-3603.
[12]	Cheng K, Zhang Y, He X, et al. Skeleton-based action reco-gnition with shift graph convolutional network[C]. Seattle: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:183-192.
[13]	Song Y F, Zhang Z, Shan C, et al. Constructing stronger and faster baselines for skeleton-based action recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(2):1474-1488.
[14]	苏波, 柴自强, 王莉, 等. 基于姿态估计的八段锦序列动作识别与评估[J]. 电子科技, 2022, 35(12):84-90.
	Su Bo, Chai Ziqiang, Wang Li, et al. Eight-section brocade sequence action recognition and evaluation based on poseestimation[J]. Electronic Science and Technology, 2022, 35(12):84-90.
[15]	Newell A, Yang K, Deng J. Stacked hourglass networks for human pose estimation[C]. Amsterdam:Computer Vision-ECCV: The Fourteenth European Conference,2016:483-499.
[16]	Toshev A, Szegedy C. Deeppose:Human pose estimation via deep neural networks[C]. Columbus: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2014:1653-1660.
[17]	Wei S E, Ramakrishna V, Kanade T, et al. Convolutional pose machines[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016: 4724-4732.
[18]	Cheng B, Xiao B, Wang J, et al. Higherhrnet: Scaleaware representation learning for bottomup human pose estimation[C]. Seattle: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:5386-5395.
[19]	Geng Z, Sun K, Xiao B, et al. Bottomup human pose estimation via disentangled keypoint regression[C]. Online: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:14676-14686.
[20]	Sun K, Xiao B, Liu D, et al. Deep high-resolution representation learning for human pose estimation[C]. Los Angeles: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:5693-5703.
[21]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Salt Lake City: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2018:7132-7141.
[22]	Luo Z, Wang Z, Huang Y, et al. Rethinking the heatmap regression for bottomup human pose estimation[C]. Seattle: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:13264-13273.

评价指标	DEKR+Transformer	DEKR+CBAM	HrHRNet(W48)^[22]	HrHRNet+Adaptive^[22]	基准网络^[19]	本文方法
AP	0.670	0.677	0.676	0.675	0.670	0.683
AP50	0.846	0.856	0.874	0.861	0.854	0.858
AP75	0.726	0.734	0.726	0.726	0.724	0.738
AP(E)	0.751	0.755	0.758	0.755	0.755	0.764
AP(M)	0.680	0.687	0.681	0.682	0.680	0.693
AP(H)	0.568	0.578	0.589	0.582	0.569	0.583

评价指标	DEKR+Transformer	DEKR+CBAM	HrHRNet(W48)^[22]	HrHRNet+Adaptive^[22]	基准网络^[19]	本文方法
AP	0.654	0.658	0.659	0.649	0.657	0.668
AP50	0.855	0.861	0.864	0.845	0.857	0.866
AP75	0.709	0.709	0.706	0.696	0.704	0.720
AP(E)	0.721	0.723	0.733	0.727	0.730	0.739
AP(M)	0.664	0.667	0.665	0.655	0.664	0.676
AP(H)	0.566	0.575	0.579	0.561	0.575	0.586

评价指标	基准网络	基准网络+SE	基准网络+weight	本文方法
AP	0.670	0.677	0.676	0.683
AP50	0.854	0.855	0.856	0.858
AP75	0.724	0.731	0.731	0.738
AP(E)	0.755	0.763	0.756	0.764
AP(M)	0.680	0.686	0.685	0.693
AP(H)	0.569	0.575	0.576	0.583

评价指标	基准网络	基准网络+SE	基准网络+weight	本文方法
AP	0.657	0.666	0.658	0.668
AP50	0.857	0.862	0.856	0.866
AP75	0.704	0.715	0.706	0.720
AP(E)	0.730	0.739	0.730	0.739
AP(M)	0.664	0.673	0.666	0.676
AP(H)	0.575	0.581	0.579	0.586