基于姿态估计的八段锦序列动作识别与评估

doi:10.16180/j.cnki.issn1007-7820.2022.12.012

Abstract

Abstract:

Action evaluation and feedback can assist fitness exercisers to improve exercise benefits effectively. In order to realize the automatic quantitative evaluation of eight-section brocade movement, a method of recognition and evaluation of human body sequence movements is proposed. The pose estimation algorithm OpenPose is used to extract the coordinates of the key points of the human body, and then normalize them and eliminate redundant points. According to the characteristics of the action, the feature vector of the fusion key points position, distance, joint angle and key points speed is constructed, and the multi-layer perceptron is employed to train the action classification. The accuracy of action recognition on the KTH and self-made eight-section brocade data sets attains to 96.7% and 98.7%, respectively. Based on the eight-section brocade action recognition results, an action sequence is constructed, and the dynamic time warping algorithm is used to calculate the similarity of the two groups of eight-section brocade action sequences. The comparative experimental results show that the similarity can effectively evaluate the integrity and synchronization of actions.

Key words: pose estimation, OpenPose, action recognition, feature extraction, feature fusion, dynamic time warping, eight-section brocade, action evaluation

CLC Number:

TP391

SU Bo,CHAI Ziqiang,WANG Li,CUI Shuaihua. Eight-Section Brocade Sequence Action Recognition and Evaluation Based on Pose Estimation[J].Electronic Science and Technology, 2022, 35(12): 84-90.

Figures/Tables 22

Figure 1.

Figure 2.

Table 1.

Figure 3.

Table 2.

Figure 4.

Figure 5.

Table 3.

Figure 6.

Table 4.

Table 5.

Figure 7.

Figure 8.

Figure 9.

Table 6.

Figure 10.

Table 7.

Figure 11.

Figure 12.

Figure 13.

Figure 14.

Table 8.

References 18

[1]	Choudhury N A, Moulik S, Choudhury S. Cloud-based real-time and remote human activity recognition system using wearable sensors[C]. Taoyuan: Proceedings of the IEEE International Conference on Consumer Electronics-Taiwan, 2020.
[2]	Tao L. Intelligent recognition method of human motion global features based on kinect skeleton information[C]. Harbin:Proceedings of the IEEE International Conference on Industrial Application of Artificial Intelligence, 2020.
[3]	Ma H, Liu H. Research on human motion recognition system based on MEMS sensor network[C]. Chengdu: Proceedings of IEEE the Fourth Advanced Information Technology, Electronic and Automation Control Conference, 2019.
[4]	Cui M M. Fang J D, Zhao Y D. Emotion recognition of human body's posture in open environment[C]. Hefei: Proceedings of the Chinese Control and Decision Conference, 2020.
[5]	Wei S E, Ramakrishna V, Kanade T, et al. Convolutional pose machines[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[6]	Batchuluun G, Kang J K, Nguyen D T, et al. Action recognition from thermal videos using joint and skeleton information[J]. IEEE Access, 2021(9):11716-11733.
[7]	Cao Z, Simon T, Wei S E, et al. Realtime multi-person 2D pose estimation using part affinity fields[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[8]	Cao Z, Hidalgo G, Simon T, et al. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(1):172-186. doi: 10.1109/TPAMI.2019.2929257
[9]	Fang H S, Xie S, Tai Y W, et al. RMPE: Regional multi-person pose estimation[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision, 2017.
[10]	Chen Y, Wang Z, Peng Y, et al. Cascaded pyramid network for multi-person pose estimation[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[11]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]. San Diego: Proceedings of the Third International Conference on Learning Representations, 2015.
[12]	Schuldt C, Laptev I, Caputo B. Recognizing human actions: A local SVM approach[C]. Cambridge: Proceedings of the Seventeenth International Conference on Pattern Recognition, 2004.
[13]	Turk M A, Pentland A. Face recognition using eigenfaces[J]. Journal of Cognitive Neuroscience, 2012, 3(1):71-86. doi: 10.1162/jocn.1991.3.1.71
[14]	Megrhi S, Jmal M, Souidene W, et al. Spatio-temporal action localization and detection for human action recognition in big dataset[J]. Journal of Visual Communication and Image Representation, 2016, 4(1):375-390.
[15]	鹿天然, 于凤芹, 陈莹. 一种基于线性序列差异分析降维的人体行为识别方法[J]. 计算机工程, 2019, 45(3):237-241.
	Lu Tianran, Yu Fengqin, Chen Ying. A human action recognition method based on LSDA reduction[J]. Computer Engineering, 2019, 45(3):237-241.
[16]	刘命强. 基于深度学习的人体动作识别研究[D]. 开封: 河南大学, 2018.
	Liu Mingqiang. Research on human action recognition based on deep learning[D]. Kaifeng: Henan University, 2018.
[17]	Sakoe H, Chiba S. Dynamic programming algorithm optimization for spoken word recognition[J]. IEEE Transactions on Acoustics Speech and Signal Processing, 1978, 26(1):43-49. doi: 10.1109/TASSP.1978.1163055
[18]	黄东方, 杨晶东. 基于改进ND-DTW算法的动态手势识别[J]. 电子科技, 2017, 30(3):37-40.
	Huang Dongfang, Yang Jingdong. Dynamic gesture recognition based on improved N-dimensional dynamic time warping[J]. Electronic Science and Technology, 2017, 30(3):37-40.

序号	部位	序号	部位
0	鼻子	9	右膝
1	脖子	10	右踝
2	右肩	11	左髋
3	右肘	12	左膝
4	右手腕	13	左踝
5	左肩	14	右眼
6	左肘	15	左眼
7	左手腕	16	右耳
8	右髋	17	左耳

关节距离L	说明	关节角θ	说明
L₁	d_(1-3)	θ₁	∠(1-2-3)
L₂	d_(2-4)	θ₂	∠(2-3-4)
L₃	d_(1-6)	θ₃	∠(1-5-6)
L₄	d_(5-7)	θ₄	∠(5-6-7)
L₅	d_(1-9)	θ₅	∠(1-8-9)
L₆	d_(8-10)	θ₆	∠(8-9-10)
L₇	d_(1-12)	θ₇	∠(1-11-12)
L₈	d_(11-13)	θ₈	∠(11-12-13)
L₉	d_(2-8)	θ₉	∠(2-1-8)
L₁₀	d_(5-11)	θ₁₀	∠(5-1-11)
L₁₁	d_(8-13)	θ₁₁	∠(8-9-13)
L₁₂	d_(10-11)	θ₁₂	∠(10-12-11)

特征类型	维度大小	说明
关键点位置	N×14×2	14个关键点的归一化坐标
关键点间距	N×12	部分关键点之间的欧式距离
关节角度	N×12	部分关节之间所呈的夹角
关键点速度	(N-1)×14×2	14个关键点x与y方向的速度

组别	特征组合类型	特征初始维数	平均准确率/%
第1组	空间特征	520	87.5
第2组	运动特征	252	71.8
第3组	融合特征	772	94.6

方法	准确率/%
光流+SURF^[14]	94.9
DT+SVM^[15]	95.4
CNN^[16]	92.2
本文(融合特征)	96.7

Eight-Section Brocade Sequence Action Recognition and Evaluation Based on Pose Estimation

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 22

References 18

Related Articles 15

Metrics

Comments

Recommended 0

	d₀₁	d₀₂	d₀₃
K=5	17.66	24.45	26.38
K=10	2.83	16.79	18.11
K=15	3.16	12.96	16.09
K=20	4.89	9.64	12.56

[1]	MA Zhuang, GAN Kaiyu, YIN Zhong. Emotion Recognition Based on Multimodal Fusion of the EEG and Peripheral Physiological Signals [J]. Electronic Science and Technology, 2025, 38(2): 62-69.
[2]	LAI Ying, JU Zhiyong, YE Yuxin. A Vehicle Detection Algorithm Based on Improved YOLOv4 [J]. Electronic Science and Technology, 2025, 38(1): 81-87.
[3]	WU Wei, MA Longhua, ZHAO Xianghong. Depression Detection Based on Cross User Audio Domain Adaptation Network [J]. Electronic Science and Technology, 2025, 38(1): 88-94.
[4]	XIE Xijun, LI Feifei. Non-Local Support Attention Network for Few-Shot Object Detection [J]. Electronic Science and Technology, 2024, 37(8): 75-83.
[5]	WANG Shengxiong, LIU Ruian, YAN Da. Image Style Transfer Algorithm Based on Improved Generative Adversarial Network [J]. Electronic Science and Technology, 2024, 37(6): 36-43.
[6]	YE Yuxin, JU Zhiyong, LAI Ying. Traffic Sign Detection Algorithm Incorporating Receptive Field Enhancement Module and Attention Mechanism [J]. Electronic Science and Technology, 2024, 37(6): 8-16.
[7]	ZHU Jinkai, FANG Lanting, JI Xiaowen, HUANG Jie. Multimodal Android Malware Detection Method Based on Behavioral and Semantic Characteristics [J]. Electronic Science and Technology, 2024, 37(5): 71-78.
[8]	PANG Jiangfei, SUN Zhanquan. Multi-Encoder Transformer for End-to-End Speech Recognition [J]. Electronic Science and Technology, 2024, 37(4): 1-7.
[9]	QIN Xiaofei, HE Wen, BAN Dongxian, GUO Hongyu, YU Jing. Research on Fast 3D Hand Keypoint Detection Algorithm Based on Anchor [J]. Electronic Science and Technology, 2024, 37(4): 77-86.
[10]	MA Wenjie, ZHANG Xuanxiong. Research on Blind Roads and Obstacle Recognition Algorithm Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(3): 75-83.
[11]	TIAN Zhixin,XU Zhen,MAO Jian,LIN Binbin,LIAO Wei. Classification Method of Steel Surface Defects Based on Multi-Scale Feature Fusion [J]. Electronic Science and Technology, 2024, 37(2): 87-95.
[12]	XIN Tenghao, LI Feifei. Research on Action Recognition Method Based on Deep Learning [J]. Electronic Science and Technology, 2024, 37(10): 64-70.
[13]	ZHANG Hui, LI Feifei. Emotion Recognition Algorithm Based on Multimodal Cross-Interaction [J]. Electronic Science and Technology, 2024, 37(10): 81-87.
[14]	ZHA Junwei,ZHANG Hongyan. Dynamic Receptive Field Feature Selection Dehazing Network [J]. Electronic Science and Technology, 2023, 36(7): 56-63.
[15]	CUI Zhuodong,CHEN Wei,YIN Zhong. Helmet Wearing Detection Based on Enhanced Feature Fusion Network [J]. Electronic Science and Technology, 2023, 36(4): 44-51.