基于姿态估计的八段锦序列动作识别与评估

doi:10.16180/j.cnki.issn1007-7820.2022.12.012

Abstract

Abstract:

Action evaluation and feedback can assist fitness exercisers to improve exercise benefits effectively. In order to realize the automatic quantitative evaluation of eight-section brocade movement, a method of recognition and evaluation of human body sequence movements is proposed. The pose estimation algorithm OpenPose is used to extract the coordinates of the key points of the human body, and then normalize them and eliminate redundant points. According to the characteristics of the action, the feature vector of the fusion key points position, distance, joint angle and key points speed is constructed, and the multi-layer perceptron is employed to train the action classification. The accuracy of action recognition on the KTH and self-made eight-section brocade data sets attains to 96.7% and 98.7%, respectively. Based on the eight-section brocade action recognition results, an action sequence is constructed, and the dynamic time warping algorithm is used to calculate the similarity of the two groups of eight-section brocade action sequences. The comparative experimental results show that the similarity can effectively evaluate the integrity and synchronization of actions.

Key words: pose estimation, OpenPose, action recognition, feature extraction, feature fusion, dynamic time warping, eight-section brocade, action evaluation

CLC Number:

TP391

SU Bo,CHAI Ziqiang,WANG Li,CUI Shuaihua. Eight-Section Brocade Sequence Action Recognition and Evaluation Based on Pose Estimation[J].Electronic Science and Technology, 2022, 35(12): 84-90.

Figures/Tables 22

Figure 1.

Figure 2.

Table 1.

Figure 3.

Table 2.

Figure 4.

Figure 5.

Table 3.

Figure 6.

Table 4.

Table 5.

Figure 7.

Figure 8.

Figure 9.

Table 6.

Figure 10.

Table 7.

Figure 11.

Figure 12.

Figure 13.

Figure 14.

Table 8.

References 18

[1]	Choudhury N A, Moulik S, Choudhury S. Cloud-based real-time and remote human activity recognition system using wearable sensors[C]. Taoyuan: Proceedings of the IEEE International Conference on Consumer Electronics-Taiwan, 2020.
[2]	Tao L. Intelligent recognition method of human motion global features based on kinect skeleton information[C]. Harbin:Proceedings of the IEEE International Conference on Industrial Application of Artificial Intelligence, 2020.
[3]	Ma H, Liu H. Research on human motion recognition system based on MEMS sensor network[C]. Chengdu: Proceedings of IEEE the Fourth Advanced Information Technology, Electronic and Automation Control Conference, 2019.
[4]	Cui M M. Fang J D, Zhao Y D. Emotion recognition of human body's posture in open environment[C]. Hefei: Proceedings of the Chinese Control and Decision Conference, 2020.
[5]	Wei S E, Ramakrishna V, Kanade T, et al. Convolutional pose machines[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[6]	Batchuluun G, Kang J K, Nguyen D T, et al. Action recognition from thermal videos using joint and skeleton information[J]. IEEE Access, 2021(9):11716-11733.
[7]	Cao Z, Simon T, Wei S E, et al. Realtime multi-person 2D pose estimation using part affinity fields[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[8]	Cao Z, Hidalgo G, Simon T, et al. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(1):172-186. doi: 10.1109/TPAMI.2019.2929257
[9]	Fang H S, Xie S, Tai Y W, et al. RMPE: Regional multi-person pose estimation[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision, 2017.
[10]	Chen Y, Wang Z, Peng Y, et al. Cascaded pyramid network for multi-person pose estimation[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[11]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]. San Diego: Proceedings of the Third International Conference on Learning Representations, 2015.
[12]	Schuldt C, Laptev I, Caputo B. Recognizing human actions: A local SVM approach[C]. Cambridge: Proceedings of the Seventeenth International Conference on Pattern Recognition, 2004.
[13]	Turk M A, Pentland A. Face recognition using eigenfaces[J]. Journal of Cognitive Neuroscience, 2012, 3(1):71-86. doi: 10.1162/jocn.1991.3.1.71
[14]	Megrhi S, Jmal M, Souidene W, et al. Spatio-temporal action localization and detection for human action recognition in big dataset[J]. Journal of Visual Communication and Image Representation, 2016, 4(1):375-390.
[15]	鹿天然, 于凤芹, 陈莹. 一种基于线性序列差异分析降维的人体行为识别方法[J]. 计算机工程, 2019, 45(3):237-241.
	Lu Tianran, Yu Fengqin, Chen Ying. A human action recognition method based on LSDA reduction[J]. Computer Engineering, 2019, 45(3):237-241.
[16]	刘命强. 基于深度学习的人体动作识别研究[D]. 开封: 河南大学, 2018.
	Liu Mingqiang. Research on human action recognition based on deep learning[D]. Kaifeng: Henan University, 2018.
[17]	Sakoe H, Chiba S. Dynamic programming algorithm optimization for spoken word recognition[J]. IEEE Transactions on Acoustics Speech and Signal Processing, 1978, 26(1):43-49. doi: 10.1109/TASSP.1978.1163055
[18]	黄东方, 杨晶东. 基于改进ND-DTW算法的动态手势识别[J]. 电子科技, 2017, 30(3):37-40.
	Huang Dongfang, Yang Jingdong. Dynamic gesture recognition based on improved N-dimensional dynamic time warping[J]. Electronic Science and Technology, 2017, 30(3):37-40.

序号	部位	序号	部位
0	鼻子	9	右膝
1	脖子	10	右踝
2	右肩	11	左髋
3	右肘	12	左膝
4	右手腕	13	左踝
5	左肩	14	右眼
6	左肘	15	左眼
7	左手腕	16	右耳
8	右髋	17	左耳

关节距离L	说明	关节角θ	说明
L₁	d_(1-3)	θ₁	∠(1-2-3)
L₂	d_(2-4)	θ₂	∠(2-3-4)
L₃	d_(1-6)	θ₃	∠(1-5-6)
L₄	d_(5-7)	θ₄	∠(5-6-7)
L₅	d_(1-9)	θ₅	∠(1-8-9)
L₆	d_(8-10)	θ₆	∠(8-9-10)
L₇	d_(1-12)	θ₇	∠(1-11-12)
L₈	d_(11-13)	θ₈	∠(11-12-13)
L₉	d_(2-8)	θ₉	∠(2-1-8)
L₁₀	d_(5-11)	θ₁₀	∠(5-1-11)
L₁₁	d_(8-13)	θ₁₁	∠(8-9-13)
L₁₂	d_(10-11)	θ₁₂	∠(10-12-11)

特征类型	维度大小	说明
关键点位置	N×14×2	14个关键点的归一化坐标
关键点间距	N×12	部分关键点之间的欧式距离
关节角度	N×12	部分关节之间所呈的夹角
关键点速度	(N-1)×14×2	14个关键点x与y方向的速度

组别	特征组合类型	特征初始维数	平均准确率/%
第1组	空间特征	520	87.5
第2组	运动特征	252	71.8
第3组	融合特征	772	94.6

方法	准确率/%
光流+SURF^[14]	94.9
DT+SVM^[15]	95.4
CNN^[16]	92.2
本文(融合特征)	96.7

Eight-Section Brocade Sequence Action Recognition and Evaluation Based on Pose Estimation

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 22

References 18

Related Articles 15

Metrics

Comments

Recommended 0

	d₀₁	d₀₂	d₀₃
K=5	17.66	24.45	26.38
K=10	2.83	16.79	18.11
K=15	3.16	12.96	16.09
K=20	4.89	9.64	12.56

[1]	ZHANG Xindong,FU Dongxiang. Glasses Virtual Try-on System Based on 3D Frame Modeling [J]. Electronic Science and Technology, 2022, 35(9): 52-57.
[2]	DENG Yuan,SHI Yiping,JIANG Yueying,ZHU Yamei,LIU Jin. Infant Expression Recognition Algorithm Based on MobileNetV2 and LBP Feature Fusion [J]. Electronic Science and Technology, 2022, 35(8): 47-52.
[3]	ZHANG Qiaomu,ZHONG Qianwen,SUN Ming,LUO Wencheng,CHAI Xiaodong. Research on Dynamic Monitoring Method of Pantograph-Net Contact Position in Complex Environment [J]. Electronic Science and Technology, 2022, 35(8): 66-72.
[4]	SUN Kang,XUAN Xuyang,LIU Penghui,ZHAO Laijun,LONG Jie. Partial Discharge Pattern Recognition of Cable Based on CNN-DCGAN under Small Data [J]. Electronic Science and Technology, 2022, 35(7): 7-13.
[5]	SHEN Ningjing,YUAN Jian. Crowd Counting Algorithm Based on Residual Dense Connection and Attention Fusion [J]. Electronic Science and Technology, 2022, 35(6): 6-12.
[6]	Hongjun SAN,Wanglin WANG,Jiupeng CHEN,Feiya XIE,Yangyang XU,Jia CHEN. VSLAM for Indoor Dynamic Scenes [J]. Electronic Science and Technology, 2022, 35(4): 14-19.
[7]	Lingyu JI,Yongbin GAO,Chenglu ZHAO,Xianhua TANG,Kaicheng XU,Jiacheng XU. CTA Segmentation Algorithm of Abdominal Artery Based on 3D Fully Convolutional Network [J]. Electronic Science and Technology, 2022, 35(3): 38-44.
[8]	WU Weijia,YANG Jian,YUAN Tianchen,SHAO Zhihui. Research on Track Structure Damage Identification Based on Support Vector Machine [J]. Electronic Science and Technology, 2022, 35(2): 27-33.
[9]	LI Hui,WANG Yicheng. CNNCIFG-Attention Model for Text Sentiment Classifcation [J]. Electronic Science and Technology, 2022, 35(2): 46-51.
[10]	SHAO Zhihui,YANG Jian,YUAN Tianchen,WU Weijia. Sleeper Diseases Diagnosis Based on Permutation Entropy and Support Vector Machine [J]. Electronic Science and Technology, 2022, 35(2): 52-58.
[11]	ZUO Wencheng,ZHAO Ziwen,XU Zhijiang,TAN Kangbo. Analysis and Research of Electromagnetic Environment in Space Station Cabin Based on 5G Communication [J]. Electronic Science and Technology, 2022, 35(10): 1-7.
[12]	ZHAO Chong,CHI Mengmeng,CHU Cong,ZHANG Peng. Research on Motion Simulation and Visual Recognition Algorithm of Guide Dog Walking Mechanism [J]. Electronic Science and Technology, 2021, 34(9): 66-72.
[13]	WANG Qingwen,HU Haiyang. Workflow Recognition Based on Temporal Action Detection [J]. Electronic Science and Technology, 2021, 34(8): 14-18.
[14]	SI Mingming,CHEN Wei,HU Chunyan,YIN Zhong. Fundus Blood Vessel Image Segmentation Combining Resnet50 and U-Net [J]. Electronic Science and Technology, 2021, 34(8): 19-24.
[15]	MA Lixin,DOU Chenfei,SONG Chencan,YANG Tianxiao. Insulator Nondestructive Testing Based on Feature Fusion CNN [J]. Electronic Science and Technology, 2021, 34(7): 26-30.