球类运动中人体姿态估计研究进展

doi:10.16180/j.cnki.issn1007-7820.2023.01.005

Abstract

Abstract:

Human pose estimation usually uses a single RGB image to locate the key points of human body to estimate the position of human body and joint points. Ball games are usually regarded as fast sports, and errors cannot be avoided in judging the technical legitimacy of players by subjective observation. Therefore,based on the estimation of human body posture, the athlete posture analysis technology is used to assist training and penalty. This method effectively avoids the traditional system positioning the athlete posture due to human subjective judgment error. At present, the research of human pose estimation can be divided into traditional algorithm and deep learning algorithm. Based on the deep learning algorithm, it can be divided into single person pose detection and multi person pose detection.Through the construction of neural network,human pose estimation based on deep learning algorithm uses machine learning method to extract image features and read image information,and perform performance comparison and analysis on mainstream data sets for human pose estimation. The application of human body posture estimation in ball games can provide scientific reference for athletes' daily training, and also ensure the fairness and justice of athletes in the game to the greatest extent.

Key words: human pose estimation, video recognition, ball game, key point positioning, feature extraction, neural network, target detection, supplementary training

CLC Number:

TP751

ZHANG Manjie,YANG Fangyan,JI Yunfeng. Research Progress of Body Posture Estimation in Ball Games[J].Electronic Science and Technology, 2023, 36(1): 28-37.

Figures/Tables 11

Table 1.

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Table 2.

Table 3.

Table 4.

References 48

[1]	周瑾, 孟祥印, 叶美松. 基于计算机视觉和LabVIEW平台的网球鹰眼系统[J]. 传感器与微系统, 2018, 37(7):102-104.
	Zhou Jin, Meng Xiangyin, Ye Meisong. Tennis hawkeye system based on computer vision and LabVIEW platform[J]. Transducer and Microsystem Technologies, 2018, 37(7):102-104.
[2]	Lai J H, Chien S Y. Tennis video with semantic scalability[C]. San Deigo: Proceedings of the IEEE International Symposium on Multimedia, 2009.
[3]	Eltoukhy M, Asfour S, Thompson C, et al. Evaluation of the performance of digital video analysis of human motion: Dartfish tracking system[J]. International Journal of Scientific and Engineering Research, 2012, 3(3):1-6.
[4]	于俊清, 王宁. 基于子窗口区域的足球视频镜头分类[J]. 中国图象图形学报, 2008, 13(7):152-157.
	Yu Junqing, Wang Ning. Shot classification for soccer video based on sub-window region[J]. Journal of Image and Graphics, 2008, 13(7):152-157.
[5]	Hanna J, Patlar F, Akbulut A, et al. HMM based classification of sports videos using color feature[C]. Sofia: Proceedings of the IEEE International Conference Intelligent Systems, 2012.
[6]	朱映映, 刘剑武, 宋娜. 篮球视频冗余数据分析与检测[J]. 小型微型计算机系统, 2010, 31(9):1873-1876.
	Zhu Yingying, Liu Jianwu, Song Na. Analysis and detection of redundancy data on basketball video[J]. Journal of Chinese Computer Systems, 2010, 31(9):1873-1876.
[7]	Pandya D S, Zaveri M A. A novel framework for semantic analysis of an illumination-variant soccer video[J]. EURASIP Journal on Image and Video Processing, 2014, 2014(1):49-56. doi: 10.1186/1687-5281-2014-49
[8]	Baysal S, Duygulu P. Sentioscope: A soccer player tracking system using model field particles[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2016, 26(7):1350-1362. doi: 10.1109/TCSVT.2015.2455713
[9]	Wang Z, Yu J, He Y. Soccer video event annotation by synchronization of attack-defense clips and match reports with coarse-grained time information[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017, 27(5):1104-1117. doi: 10.1109/TCSVT.2016.2515280
[10]	Yoon Y, Hwang H, Choi Y, et al. Analyzing basketball movements and pass relationships using realtime object tracking techniques based on deep learning[J]. IEEE Access, 2019, 7(99):56564-56576. doi: 10.1109/ACCESS.2019.2913953
[11]	Chen H T, He Y Z, Hsu C C. Computer-assisted yoga training system[J]. Multimedia Tools and Applications, 2018, 77(18):23969-23991. doi: 10.1007/s11042-018-5721-2
[12]	Dan Z, Einfalt M, Eggert C, et al. Kinematic pose rectification for performance analysis and retrieval in sports[C]. Salt Lake: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018.
[13]	Kong L, Huang D, Qin J, et al. A joint framework for athlete tracking and action recognition in sports videos[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(2):532-548. doi: 10.1109/TCSVT.2019.2893318
[14]	Wu E, Koike H. FuturePong: Real-time table tennis trajectory forecasting using pose prediction network[C]. Honolulu: Proceedings of the Conference on Human Factors in Computing Systems, 2020.
[15]	Fischler M A, Elschlager R A. The representation and matching of pictorial structures[J]. IEEE Transactions on Computers, 1973, 22(1):67-92.
[16]	Dalal N, Triggs B. Histograms of oriented gradients for human detection[C]. San Diego: Proceedings of the International Conference on Computer Vision and Pattern Recognition, 2005.
[17]	Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110. doi: 10.1023/B:VISI.0000029664.99615.94
[18]	Yi Y, Ramanan D. Articulated pose estimation with flexible mixtures-of-parts[C]. Colorado Springs: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2011.
[19]	Toshev A, Szegedy C. DeepPose: Human pose estimation via deep neural networks[C]. Portland: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013.
[20]	Carreira J, Agrawal P, Fragkiadaki K, et al. Human pose estimation with iterative error feedback[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[21]	Tompson J, Jain A, Lecun Y, et al. Joint training of a convolutional network and a graphical model for human pose estimation[C]. Columbus: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[22]	Li S Z. Markov random field modeling in image analysis[M]. New York: Springer-Verlag, 2001.
[23]	Newell A, Yang K, Jia D. Stacked hourglass networks for human pose estimation[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[24]	Yang W, Li S, Ouyang W, et al. Learning feature pyramids for human pose estimation[C]. Venice: Proceedings of the IEEE International Conference on Computer Vision, 2017.
[25]	Sun K, Xiao B, Liu J, et al. Deep high-resolution representation learning for human pose estimation[C]. Long Beach: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.
[26]	Zhang J, Chen Z, Tao D. Towards high performance human keypoint detection[C]. Seattle: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[27]	Artacho B, Savakis A. UniPose: Unified human pose estimation in single images and videos[C]. Seattle: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[28]	Bulat A, Tzimiropoulos G. Human pose estimation via convolutional part heatmap regression[C]. Amsterdam: Proceedings of the European Conference on Computer Vision, 2016.
[29]	Fa N X, Kang Z, Lin Y, et al. Combining local appearance and holistic view: Dual-source deep neural networks for human pose estimation[C]. Boston: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[30]	Papandreou G, Zhu T, Kanazawa N, et al. Towards accurate multi-person pose estimation in the wild[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[31]	Ren S, He K, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. doi: 10.1109/TPAMI.2016.2577031 pmid: 27295650
[32]	Chen Y, Wang Z, Peng Y, et al. Cascaded pyramid network for multi-person pose estimation[C]. Salt Lake: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[33]	Wang M, Tighe J, Modolo D. Combining detection and tracking for human pose estimation in videos[C]. Seattle: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[34]	Huang J, Zhu Z, Guo F, et al. The devil is in the details: Delving into unbiased data processing for human pose estimation[C]. Seattle: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[35]	Yan Y, Li J, Qin J, et al. Anchor-free person search[C]. Nashville: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[36]	Ma X, Su J, Wang C, et al. Context modeling in 3D human pose estimation: A unified perspective[C]. Nashville: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[37]	Cao Z, Simon T, Wei S E, et al. Realtime multi-person 2D pose estimation using part affinity fields[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[38]	Xia F, Peng W, Chen X, et al. Joint multi-person pose estimation and semantic part segmentation[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[39]	Cheng B, Xiao B, Wang J, et al. HigherHRNet: Scale-Aware representation learning for bottom-up human pose estimation[C]. Seattle: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[40]	Fabbri M, Lanzi F, Calderara S, et al. Compressed volumetric heatmaps for multi-person 3D pose estimation[C]. Seattle: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
[41]	Luo Z, Wang Z, Huang Y, et al. Rethinking the heatmap regression for bottom-up human pose estimation[C]. Nashville: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[42]	叶飞, 刘子龙. 基于改进YOLOv3算法的行人检测研究[J]. 电子科技, 2021, 34(1):5-9.
	Ye Fei, Liu Zilong. Pedestrian detection based on improved YOLOv3 algorithm[J]. Electronic Science and Technology, 2021, 34(1):5-9.
[43]	Johnson S, Everingham M. Clustered pose and nonlinear appearance models for human pose estimation[C]. Aberystwyth: Proceedings of the British Machine Vision Conference, 2010.
[44]	Sapp B, Taskar B. MODEC: Multimodal decomposable models for human pose estimation[C]. Portland: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013.
[45]	Andriluka M, Pishchulin L, Gehler P, et al. Human pose estimation: New benchmark and state of the art analysis[C]. Columbus: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[46]	Lin T Y, Maire M, Belongie S, et al. Microsoft COCO: Common objects in context[C]. Zurich: Proceedings of the European Conference on Computer Vision, 2014.
[47]	Wu J, Zheng H, Zhao B, et al. AI Challenger: A large-scale dataset for going deeper in image understanding[C]. Honolulu: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[48]	Li J, Wang C, Zhu H, et al. CrowdPose: Efficient crowded scenes pose estimation and a new benchmark[C]. Long Beach: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.

研究来源	年份	运动类型	检测对象	目标
文献[4]	2008	足球	颜色特征	镜头分类
文献[5]	2012	足球、高尔夫球	颜色特征	视频序列分类
文献[6]	2013	篮球	颜色特征和区域特征	视频分割
文献[7]	2014	足球	光流和颜色特征	视频分割、事件分类
文献[8]	2016	足球	外观和运动模型	遮挡、多目标追踪
文献[9]	2017	足球	视频和外部文本信息	利用高级特征分析视频
文献[10]	2019	篮球	视频分类和上下文信息	球员动作跟踪与分析
文献[11]	2018	篮球	文本和视频帧	提取慢动作
文献[12]	2018	游泳	运动员关节	矫正运动员姿态
文献[13]	2019	-	运动员姿态	运动员动作识别
文献[14]	2020	乒乓球	球员2D姿态	预测乒乓球落脚点

年份	数据集	样本数量	样本标注特征	检测场景
2010	LSP	训练:1 000 测试:1 000	标注14个关节点,包含2 000个姿势注释	单人
2013	FILC	训练:3 987 测试:1 016	10个上半身关节点	单人
2014	MPII	2.5×10⁴	16个关节点,涵盖410项人类活动,包含超过4万人	单人/多人
2014	MSCOCO	超过3.3×10⁵	17个关节点,包含10万人	多人
2017	AI Challenger	训练:2.1×10⁵ 验证:3×10⁴ 测试:3×10⁴	标注14个关节点,是目前最大的人体姿态估计图像数据集	多人
2019	Crowd Pose	训练:10×10³ 验证:2×10³ 测试:8×10³	标注14个关节点,包含8万行人,适应密集场景	多人

方法	网络			表现/%
				FLIC				LSP(PCP)	MPII(PCKh@0.5)
				Eblow		Wrist		LSP(PCP)	MPII(PCKh@0.5)
基于坐标回归	Deep-Pose	Toshev	92.3		82.0		61.0		-
基于坐标回归	IEF	Carreira	-		-		72.5		81.3
基于热图检测	CNN和图模型	Tompson	95.2		91.2		67.2		-
	堆叠沙漏网络	Newell	99.0		97.0		-		90.9
	PRMs	Yang	-		-		93.9		92.0
	HRNet	Sun	-		-		-		92.3
	WASP	UniPose					72.8		92.7
回归与检测混合模型	Coordinate Net和Heatmap Net串联	Bulat	-		-		90.7		89.7
回归与检测混合模型	DS-CNN	Fan	-		-		84.0		-

方法	文献	多人姿态估计方法	主要网络	表现/%
方法	文献	多人姿态估计方法	主要网络	MPII(PCKh@0.5)	MSCOCO	MAP
自顶向下	[30]	RCNN	RetNet-101	-	-	64.9
	[32]	CPN	GLobalNet、Refinenet	-	72.1	-
	[33]	HR-NET	3D HR-NET	-	-	83.8
	[35]	Faster-RCNN	AlignPSt	-	-	94.0
	[37]	Open Pose	CPM	75.6	61.8	-
	[38]	CRF	FCN	-	-	79.1
自底向上	[39]	HR-NET	HigherHR-NET	-	74.9	-
自底向上	[41]	HRNet	W48	-	77.7	-

Research Progress of Body Posture Estimation in Ball Games

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 48

Related Articles 15

Metrics

Comments

Recommended 10

[1]	YU Qiongfang,NIU Dongyang. Mixed Prediction of Mine Pressure Time and Space Based on LSTM Network [J]. Electronic Science and Technology, 2023, 36(2): 67-72.
[2]	ZUO Bin,LI Feifei. An Effective Segmentation Method for COVID-19 CT Image Based on Attention Mechanism and Inf-Net [J]. Electronic Science and Technology, 2023, 36(2): 22-28.
[3]	CHENG Changwen,CHEN Wei,CHEN Jinhong,YIN Zhong. YOLO-Improve Detection Method of Real-Time Mask Wearing [J]. Electronic Science and Technology, 2023, 36(2): 73-80.
[4]	HUANG Yajing,LIAO Aihua,YU Miao,LI Xiaolong,HU Dingyu. An Improved CNN Method for Bearing Acoustic Fault Diagnosis [J]. Electronic Science and Technology, 2023, 36(1): 75-80.
[5]	ZHOU Yongchang,HUANG Yayu. Establishment of a Predictive Model of the Process Parameters of Secondary Moisturizing Based on BP Neural Network [J]. Electronic Science and Technology, 2022, 35(9): 79-86.
[6]	BI Jiazhen,SHEN Tuo,ZHANG Xuanxiong. A Research on Distance Measurement Between Trains in Rail Transit Based on Machine Vision [J]. Electronic Science and Technology, 2022, 35(9): 37-43.
[7]	LIU Guohua,LU Hongmin,CHEN Chongchong,LI Wanyu,WAN Jianpeng. Wideband Nonlinear Behavior Modeling of Receiver with Neural Network [J]. Electronic Science and Technology, 2022, 35(8): 1-6.
[8]	DENG Yuan,SHI Yiping,JIANG Yueying,ZHU Yamei,LIU Jin. Infant Expression Recognition Algorithm Based on MobileNetV2 and LBP Feature Fusion [J]. Electronic Science and Technology, 2022, 35(8): 47-52.
[9]	ZHANG Qiaomu,ZHONG Qianwen,SUN Ming,LUO Wencheng,CHAI Xiaodong. Research on Dynamic Monitoring Method of Pantograph-Net Contact Position in Complex Environment [J]. Electronic Science and Technology, 2022, 35(8): 66-72.
[10]	TONG Xiaosen,YANG Jinxian. Drilling Tool Acceleration Denoising Based on GRNN Network Adaptive Filtering [J]. Electronic Science and Technology, 2022, 35(7): 46-51.
[11]	SUN Kang,XUAN Xuyang,LIU Penghui,ZHAO Laijun,LONG Jie. Partial Discharge Pattern Recognition of Cable Based on CNN-DCGAN under Small Data [J]. Electronic Science and Technology, 2022, 35(7): 7-13.
[12]	ZHANG Chongchong,HUANG Yayu. A GA-BP Neural Network for Predicting the Structure of Leaf Tobacco [J]. Electronic Science and Technology, 2022, 35(6): 35-42.
[13]	SHEN Ningjing,YUAN Jian. Crowd Counting Algorithm Based on Residual Dense Connection and Attention Fusion [J]. Electronic Science and Technology, 2022, 35(6): 6-12.
[14]	WANG Peiyu,MA Lixin. Research on Permanent Magnet Synchronous Motor Servo System Based on Fuzzy Neural Network [J]. Electronic Science and Technology, 2022, 35(6): 83-88.
[15]	CHENG Shunda,CHENG Ying,SUN Shijiang. Tumor Intelligent Auxiliary Diagnosis Method Based on Machine Learning [J]. Electronic Science and Technology, 2022, 35(5): 56-59.