基于深度学习的人体骨架动作识别

doi:10.16180/j.cnki.issn1007-7820.2020.11.015

摘要/Abstract

摘要：

基于人体骨架的动作识别具有鲁棒性和视点不变性的优点,为进一步提高骨架动作识别的识别率,打破以往大部分基于深度学习的方法的输入内容为人体关节坐标的局限性,文中提出一种将几何特征与LSTM网络结合的人体骨架动作识别算法。该算法选择基于关节与选定直线之间距离的几何特征作为网络的输入,引入了一种时间选择LSTM网络进行训练。利用时间选择LSTM网络拥有选出最具识别性时间段特征的能力,在SBU Interaction数据集和UT Kinect数据集上分别取得了99.36%和99.20%的识别率。实验结果证明了该方法对人体骨架动作识别的有效性。

关键词: 动作识别, 人体骨架, 深度学习, 几何特征, 时间选择, LSTM网络

Abstract:

Based on the advantages of robustness and view-invariant representation, a skeleton-based action recognition algorithm combining geometric features with LSTM network is proposed to further improve the recognition rate and to break the limitation that the inputs of most methods based on deep learning are human joint coordinates. The geometric features based on the distances between joints and selected lines are selected as the input of the network. Then, time-selective LSTM network is introduced to train. Time selection LSTM network has the ability to select the most recognizable time period features. By using this feature, 99.36% and 99.20% recognition rates are achieved on SBU Interaction dataset and UT Kinect dataset, respectively. The experimental results show that the method is effective for human skeleton-based action recognition.

Key words: action recognition, human skeleton, deep learning, geometric features, time selective modal, Long Short-term Memory networks

中图分类号:

TP391

邬倩,吴飞,骆立志. 基于深度学习的人体骨架动作识别[J]. 电子科技, 2020, 33(11): 79-83.

WU Qian,WU Fei,LUO Lizhi. Human Skeleton-based Action Recognition Based on Deep Learning[J]. Electronic Science and Technology, 2020, 33(11): 79-83.

图/表 10

图1

图2

图3

图4

图5

表1

表2

图6

图7

表3

参考文献 20

[1]	Shotton J, Sharp T, Kipman A, et al. Real-time human pose recognition in parts from single depth images[J]. Communications of the ACM, 2013,56(1):116-124.
[2]	Chen C, Zhuang Y, Nie F, et al. Learning a 3D human pose distance metric from geometric pose descriptor[J]. IEEE Transactions on Visualization & Computer Graphics, 2011,17(11):1676-1689. doi: 10.1109/TVCG.2010.272 pmid: 21173458
[3]	Vemulapalli R, Arrate F, Chellappa R. Human action recognition by representing 3D skeletons as points in a Lie group[C]. Columbus:IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[4]	Shao Z, Li Y. Integral invariants for space motion trajectory matching and recognition[J]. Pattern Recognit, 2015,48(8):2418-2432.
[5]	陈胜娣, 魏维, 何冰倩, 等. 基于改进的深度卷积神经网络的人体动作识别方法[J]. 计算机应用研究, 2019,36(4):1-7.
	Chen Shengdi, Wei Wei, He Bingqian, et al. Action recognition base on improved deep convolutional neural network[J]. Application Research of Computers, 2019,36(4):1-7.
[6]	梁玉强, 陈劲杰, 叶其含. 基于AdaBoost和BP网络的机器人动作理解[J]. 电子科技, 2017,30(8):63-66.
	Liang Yuqiang, Cheng Jinjie, Ye Qihan. Robot motion understanding based on AdaBoost and BP networks[J]. Electronic Science and Technology, 2017,30(8):63-66.
[7]	Veeriah V, Zhuang N, Qi G J. Differential recurrent neural networks for action recognition[C]. Santiago:IEEE International Conference on Computer Vision, 2015.
[8]	Liu J, Shahroudy A, Xu D, et al. Spatio-temporal LSTM with trust gates for 3D human action recognition[C]. Amsterdam:The Fourteenth European Conference on Computer Vision (ECCV), 2016.
[9]	Song S, Lan C, Xing J, et al. An end-to-end spatio-temporal attention model for human action recognition from skeleton data[C]. San Francisco:The Thirty-first AAAI Conference on Artificial Intelligence, 2017.
[10]	Zhang S, Yang Y, Xiao J, et al. Fusing geometric features for skeleton-based action recognition using multilayer LSTM networks[J]. IEEE Transactions on Multimedia, 2018,20(9):2330-2343.
[11]	Liu J, Wang G, Hu P, et al. Global context-aware attention LSTM networks for 3D action recognition[C]. Hawaii: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[12]	Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult[J]. IEEE Transactions on Neural Networks, 1994,5(2):157-166. doi: 10.1109/72.279181 pmid: 18267787
[13]	Yun K, Honorio J, Chattopadhyay D, et al. Two-person interaction detection using body-pose features and multiple instance learning[C]. Rhode Island:IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2012.
[14]	Xia L, Chen C C, Aggarwal J K. View invariant human action recognition using histograms of 3D joints[C]. Rhode Island:IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2012.
[15]	Ji Y, Ye G, Cheng H. Interactive body part contrast mining for human interaction recognition[C]. Chengdu:IEEE International Conference on Multimedia and Expo Workshops, 2014.
[16]	Li W, Wen L, Chuah M C, et al. Category-blind human action recognition: a practical recognition system[C]. Santiago:IEEE International Conference on Computer Vision, 2015.
[17]	Du Y, Wang W, Wang L. Hierarchical recurrent neural network for skeleton based action recognition[C]. Boston: IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[18]	Zhu W, Lan C, Xing J, et al. Cooccurrence feature learning for skeleton based action recognition using regularized deep LSTM networks[C]. Phoenix:The Thirtieth AAAI Conference on Artificial Intelligence, 2016.
[19]	Zhu Y, Chen W, Guo G. Fusing spatiotemporal features and joints for 3D action recognition[C]. Portland:IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013.
[20]	Anirudh R, Turaga P, Su J, et al. Elastic functional coding of human actions: from vector-fields to latent variables[C]. Boston:IEEE Conference on Computer Vision and Pattern Recognition, 2015.

方法	识别率
Yun, et al.^[13]	80.30%
Ji, et al.^[15]	86.90%
CHARM^[16]	83.90%
HBRNN-L^[17]	80.35%
Co-occurrence LSTM^[18]	90.41%
STA-LSTM^[9]	91.50%
GCA-LSTM^[11]	94.10%
ST-LSTM^[8]	93.30%
Zhang, et al.^[10]	99.02%
Our method	99.36%

方法	识别率
Skeleton Joint Features^[19]	87.90%
Elastic functional coding^[20]	94.90%
Lie Group^[3]	93.60%
ST-LSTM^[8]	95.00%
Zhang et al.^[10]	95.96%
GCA-LSTM^[11]	98.50%
Our method	99.20%