基于深度学习的人体骨架动作识别

doi:10.16180/j.cnki.issn1007-7820.2020.11.015

Abstract

Abstract:

Based on the advantages of robustness and view-invariant representation, a skeleton-based action recognition algorithm combining geometric features with LSTM network is proposed to further improve the recognition rate and to break the limitation that the inputs of most methods based on deep learning are human joint coordinates. The geometric features based on the distances between joints and selected lines are selected as the input of the network. Then, time-selective LSTM network is introduced to train. Time selection LSTM network has the ability to select the most recognizable time period features. By using this feature, 99.36% and 99.20% recognition rates are achieved on SBU Interaction dataset and UT Kinect dataset, respectively. The experimental results show that the method is effective for human skeleton-based action recognition.

Key words: action recognition, human skeleton, deep learning, geometric features, time selective modal, Long Short-term Memory networks

CLC Number:

TP391

WU Qian,WU Fei,LUO Lizhi. Human Skeleton-based Action Recognition Based on Deep Learning[J].Electronic Science and Technology, 2020, 33(11): 79-83.

Figures/Tables 10

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Table 1

Table 2

Figure 6.

Figure 7.

Table 3

References 20

[1]	Shotton J, Sharp T, Kipman A, et al. Real-time human pose recognition in parts from single depth images[J]. Communications of the ACM, 2013,56(1):116-124.
[2]	Chen C, Zhuang Y, Nie F, et al. Learning a 3D human pose distance metric from geometric pose descriptor[J]. IEEE Transactions on Visualization & Computer Graphics, 2011,17(11):1676-1689. doi: 10.1109/TVCG.2010.272 pmid: 21173458
[3]	Vemulapalli R, Arrate F, Chellappa R. Human action recognition by representing 3D skeletons as points in a Lie group[C]. Columbus:IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[4]	Shao Z, Li Y. Integral invariants for space motion trajectory matching and recognition[J]. Pattern Recognit, 2015,48(8):2418-2432.
[5]	陈胜娣, 魏维, 何冰倩, 等. 基于改进的深度卷积神经网络的人体动作识别方法[J]. 计算机应用研究, 2019,36(4):1-7.
	Chen Shengdi, Wei Wei, He Bingqian, et al. Action recognition base on improved deep convolutional neural network[J]. Application Research of Computers, 2019,36(4):1-7.
[6]	梁玉强, 陈劲杰, 叶其含. 基于AdaBoost和BP网络的机器人动作理解[J]. 电子科技, 2017,30(8):63-66.
	Liang Yuqiang, Cheng Jinjie, Ye Qihan. Robot motion understanding based on AdaBoost and BP networks[J]. Electronic Science and Technology, 2017,30(8):63-66.
[7]	Veeriah V, Zhuang N, Qi G J. Differential recurrent neural networks for action recognition[C]. Santiago:IEEE International Conference on Computer Vision, 2015.
[8]	Liu J, Shahroudy A, Xu D, et al. Spatio-temporal LSTM with trust gates for 3D human action recognition[C]. Amsterdam:The Fourteenth European Conference on Computer Vision (ECCV), 2016.
[9]	Song S, Lan C, Xing J, et al. An end-to-end spatio-temporal attention model for human action recognition from skeleton data[C]. San Francisco:The Thirty-first AAAI Conference on Artificial Intelligence, 2017.
[10]	Zhang S, Yang Y, Xiao J, et al. Fusing geometric features for skeleton-based action recognition using multilayer LSTM networks[J]. IEEE Transactions on Multimedia, 2018,20(9):2330-2343.
[11]	Liu J, Wang G, Hu P, et al. Global context-aware attention LSTM networks for 3D action recognition[C]. Hawaii: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[12]	Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult[J]. IEEE Transactions on Neural Networks, 1994,5(2):157-166. doi: 10.1109/72.279181 pmid: 18267787
[13]	Yun K, Honorio J, Chattopadhyay D, et al. Two-person interaction detection using body-pose features and multiple instance learning[C]. Rhode Island:IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2012.
[14]	Xia L, Chen C C, Aggarwal J K. View invariant human action recognition using histograms of 3D joints[C]. Rhode Island:IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2012.
[15]	Ji Y, Ye G, Cheng H. Interactive body part contrast mining for human interaction recognition[C]. Chengdu:IEEE International Conference on Multimedia and Expo Workshops, 2014.
[16]	Li W, Wen L, Chuah M C, et al. Category-blind human action recognition: a practical recognition system[C]. Santiago:IEEE International Conference on Computer Vision, 2015.
[17]	Du Y, Wang W, Wang L. Hierarchical recurrent neural network for skeleton based action recognition[C]. Boston: IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[18]	Zhu W, Lan C, Xing J, et al. Cooccurrence feature learning for skeleton based action recognition using regularized deep LSTM networks[C]. Phoenix:The Thirtieth AAAI Conference on Artificial Intelligence, 2016.
[19]	Zhu Y, Chen W, Guo G. Fusing spatiotemporal features and joints for 3D action recognition[C]. Portland:IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2013.
[20]	Anirudh R, Turaga P, Su J, et al. Elastic functional coding of human actions: from vector-fields to latent variables[C]. Boston:IEEE Conference on Computer Vision and Pattern Recognition, 2015.

方法	识别率
Yun, et al.^[13]	80.30%
Ji, et al.^[15]	86.90%
CHARM^[16]	83.90%
HBRNN-L^[17]	80.35%
Co-occurrence LSTM^[18]	90.41%
STA-LSTM^[9]	91.50%
GCA-LSTM^[11]	94.10%
ST-LSTM^[8]	93.30%
Zhang, et al.^[10]	99.02%
Our method	99.36%

方法	识别率
Skeleton Joint Features^[19]	87.90%
Elastic functional coding^[20]	94.90%
Lie Group^[3]	93.60%
ST-LSTM^[8]	95.00%
Zhang et al.^[10]	95.96%
GCA-LSTM^[11]	98.50%
Our method	99.20%

Human Skeleton-based Action Recognition Based on Deep Learning

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 20

Related Articles 15

Metrics

Comments

Recommended 10

[1]	CHENG Xiaoya,ZHANG Lei. Research on Occluded Face Recognition Method Based on Deep Learning [J]. Electronic Science and Technology, 2022, 35(1): 35-39.
[2]	ZHAO Chong,CHI Mengmeng,CHU Cong,ZHANG Peng. Research on Motion Simulation and Visual Recognition Algorithm of Guide Dog Walking Mechanism [J]. Electronic Science and Technology, 2021, 34(9): 66-72.
[3]	WANG Qingwen,HU Haiyang. Workflow Recognition Based on Temporal Action Detection [J]. Electronic Science and Technology, 2021, 34(8): 14-18.
[4]	MA Lixin,DOU Chenfei,SONG Chencan,YANG Tianxiao. Insulator Nondestructive Testing Based on Feature Fusion CNN [J]. Electronic Science and Technology, 2021, 34(7): 26-30.
[5]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.
[6]	SHAO Hang,WANG Yongxiong,QIN Yulong. Digital Image Composition Optimization Based on Salient Feature Algorithm [J]. Electronic Science and Technology, 2021, 34(3): 36-42.
[7]	YAN Chao,SUN Zhanquan,TIAN Engang,ZHAO Yangyang,FAN Xiaoyan. Research Progress of Medical Image Segmentation Based on Deep Learning [J]. Electronic Science and Technology, 2021, 34(2): 7-11.
[8]	CHENG Junhua,ZENG Guohui,LIU Jin. Research on Complex Background Image Classification Method Based on Deep Learning [J]. Electronic Science and Technology, 2020, 33(12): 59-66.
[9]	DUAN Yujia,JU Ting. Evaluation of Code Review Comments Based on Deep Learning [J]. Electronic Science and Technology, 2020, 33(1): 39-45.
[10]	TONG Zeyou,FENG Shimin,HOU Xiaoqing,DING Enjie. Recognition of Underground Miners’ Rule-Violated Behavior Based on Safety Helmet Detection [J]. Electronic Science and Technology, 2019, 32(9): 26-31.
[11]	HU Shaocong. Research on Face Recognition Based on Deep Learning [J]. Electronic Science and Technology, 2019, 32(6): 82-86.
[12]	JIANG Meng,WANG Ziniu,GAO Jianling. Chinese Word Segmentation Based on Joint Training of Heterogeneous Data [J]. Electronic Science and Technology, 2019, 32(4): 29-33.
[13]	LI Rongrui,SHI Lin,ZHAO Wei. Minority Headdress Recognition Based on Convolutional Neural Network [J]. Electronic Science and Technology, 2019, 32(2): 51-55.
[14]	MENG Xin. Research on Recognition Method of Legal Documents Based on Deep Learning [J]. Electronic Science and Technology, 2019, 32(12): 84-86.
[15]	YUAN Xiaoping,WANG Gang,WANG Yefeng,WANG Zheyuan,SUN Hui. Traffic Sign Recognition Method Based on Improved Convolutional Neural Network [J]. Electronic Science and Technology, 2019, 32(11): 28-32.