结合形变模型的人体姿态估计优化算法

doi:10.19665/j.issn1001-2400.2020.02.004

Abstract

Abstract:

An optimization algorithm is proposed utilizing the video data and point cloud data captured by the depth camera to solve the problems such as error-proneness and incoherence of motion sequence caused by the existing human pose estimation algorithms based on the morphable model. For video data, the neural network is first used in extracting the model parameters from each color image frame. Next, the human key-points and contour constraint are considered to optimize the above parameters. Then the coherence between every two consecutive frames is utilized to correct the error of pose estimation, thus making the resulting motion sequence smoother. In addition, the point cloud and the model obtained from the corresponding color image frame are used as the joint input to further improve the estimation accuracy. Finally, the distance between the point cloud and the corresponding point of the model is constrained to be as small as possible to obtain a more reasonable solution. The proposed algorithm and the state-of-the-art algorithms are compared qualitatively and quantitatively on the data set and real video set. Experimental results show that the algorithm can effectively correct the error and incoherence in the single-frame pose estimation results and greatly improve the accuracy when using point cloud data optimization.

Key words: pose estimation, motion reconstruction, morphable model, point cloud, neural network

CLC Number:

TP391

LI Jian,ZHANG Haoruo,HE Bin. Optimization algorithm for estimating the human pose by using the morphable model[J].Journal of Xidian University, 2020, 47(2): 23-31.

Figures/Tables 9

References 19

[1]	WEI S E, RAMAKRISHNA V, KANADE T , et al. Convolutional Pose Machines[C]// Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2016: 4724-4732.
[2]	FANG H S, XIE S, TAI Y W , et al. Rmpe: Regional Multi-person Pose Estimation[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017: 2353-2362.
[3]	CAO Z, SIMON T, WEI S E , et al. Realtime Multi-person 2d Pose Estimation Using Part Affinity Fields[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017: 1302-1310.
[4]	LUO Y, REN J, WANG Z , et al. LSTM Pose Machines[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2018: 5207-5215.
[5]	LOPER M, MAHMOOD N, ROMERO J , et al. SMPL:a Skinned Multi-person Linear Model[J]. ACM Transactions on Graphics, 2015,34(6):248.
[6]	BOGO F, KANAZAWA A, LASSNER C , et al. Keep It SMPL:Automatic Estimation of 3D Human Pose and Shape from a Single Image[C]// Lecture Notes in Computer Science: 9909. Heidelberg:Springer Verlag, 2016: 561-578.
[7]	LASSNER C, ROMERO J, KIEFEL M , et al. Unite the People:Closing the Loop between 3d and 2d Human RepresEntations[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017: 4704-4713.
[8]	HUANG Y, BOGO F, LASSNER C , et al. Towards Accurate Marker-less Human Shape and Pose Estimation over Time[C]// Proceedings of the 2017 International Conference on 3D Vision. Piscataway:IEEE, 2018: 421-430.
[9]	ALLDIECK T, MAGNOR M, XU W , et al. Video Based Reconstruction of 3d People Models[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2018: 8387-8397.
[10]	OMRAN M, LASSNER C, PONS-MOLL G , et al. Neural Body Fitting:Unifying Deep Learning and Model Based Human Pose and Shape Estimation[C]// Proceedings of the 2018 International Conference on 3D Vision. Piscataway:IEEE, 2018: 484-494.
[11]	KANAZAWA A, BLACK M J, JACOBS D W , et al. End-to-end Recovery of Human Shape and Pose[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2018: 7122-7131.
[12]	KOLOTOUROS N, PAVLAKOS G, DANIILIDIS K . Convolutional Mesh Regression for Single-image Human Shape Reconstruction[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019: 4501-4510.
[13]	KANAZAWA A, ZHANG J, FELSEN P , et al. Learning 3D Human Dynamics from Video[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019: 5614-5623.
[14]	CAELLES S, MANINIS K K, PONT-TUSET J , et al. One-shot Video Object Segmentation[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017: 5320-5329.
[15]	LOPER M M, BLACK M J . OpenDR:an Approximate Differentiable Renderer[C]// Lecture Notes in Computer Science: 8695. Heidelberg:Springer Verlag, 2014: 154-169.
[16]	AHOOKHOSH M, AMINI K, KIMIAEI M . A Globally Convergent Trust-region Method for Large-scale Symmetric Nonlinear Systems[J]. Numerical Functional Analysis and Optimization, 2015,36(7):830-855.
[17]	IONESCU C, PAPAVA D, OLARU V , et al. Human3.6M:Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014,36(6):1325-1339.
[18]	彭亚丽, 刘侍刚, 梁新刚 , 等. 三维变形体的因式分解重建方法[J]. 西安电子科技大学学报, 2016,43(1):116-119.
	PENG Yali, LIU Shigang, LIANG Xingang , et al. Factorization Reconstruction Method for the 3D Deformable Structure[J] . Journal of Xidian University, 2016,43(1):116-119.
[19]	PAVLAKOS G, CHOUTAS V, GHORBANI N , et al. Expressive Body Capture:3D Hands, Face, and Body from a Single Image[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019: 10975-10985.

算法	PA-MPJPE/mm
文献[6]	82.3
文献[8]	58.4
文献[10]	59.9
人体网格重构算法^[11]	56.8
文献[13]	58.1
文中算法	56.3

数据	人体网格重构算法^[11]	均方根文献[9]	点云姿态估计算法
图7第一列	0.020221	0.018260	0.011648
图7第二列	0.019626	0.025794	0.013060
图7第三列	0.026306	0.026720	0.010320
图7第四列	0.028419	0.026597	0.013047
图7第五列	0.020325	0.025399	0.014320
图7第六列	0.027542	0.033031	0.012443
平均	0.023739	0.025096	0.012473

Optimization algorithm for estimating the human pose by using the morphable model

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 19

Related Articles 15

Metrics

Comments

Recommended 0

[1]	QU Jiahui, HE Jie, DONG Wenqian, LI Yunsong, ZHANG Tongzhen, YANG Yufei. Change detection method based on multi-scale and multi-resolution information fusion [J]. Journal of Xidian University, 2025, 52(1): 105-116.
[2]	KONG Fanqiang, YU Shengjie, WANG Kun, FANG Xu, LV Zhijie. Hyperspectral image unmixing method based on convolutional recurrent neural networks [J]. Journal of Xidian University, 2025, 52(1): 142-151.
[3]	CHEN Chen, CHENG Rong, SONG Bin. End-to-end heterogeneous graph information collaborative filtering for recommendation [J]. Journal of Xidian University, 2025, 52(1): 163-180.
[4]	LI Linke, CHEN Jie, LIU Jun. Improved schemes and applications of the neural network differential distinguisher [J]. Journal of Xidian University, 2025, 52(1): 196-214.
[5]	WANG Danyang, PIAO Chunying, LIU Qi, GUAN Lei, LI Zan. CTS features based electromagnetic interference identification at radio observatory site [J]. Journal of Xidian University, 2025, 52(1): 80-93.
[6]	TANG Shuyuan, ZHOU Yiqing, LI Jintao, LIU Chang, SHI Jinglin. Dual attention pedestrian detector for occlusion scenario based on feature calibration [J]. Journal of Xidian University, 2024, 51(6): 25-39.
[7]	CAI Gushun, LIU Jinhui, ZHANG Xindan, HUANG Zhao, WANG Quan. PINN-based method for solving DC operating points in nonlinear circuits [J]. Journal of Xidian University, 2024, 51(6): 91-103.
[8]	SHI Jiaqi, YANG Minglei, LIAN Hao, YE Zhou, XU Guanghui. EVD clutter suppression method based on the self-organizing neural network [J]. Journal of Xidian University, 2024, 51(5): 46-57.
[9]	WANG Yulai, LIAO Xiaomin, HE Haiguang, YE Guojun. Knowledge graph assisted spectrum resource optimization algorithm for UAVs [J]. Journal of Xidian University, 2024, 51(5): 58-70.
[10]	LI Xiaohan, YANG Yanbo, ZHANG Jiawei, LI Baoshan, MA Jianfeng. Graph neural network vulnerability detection for ethernet smart contracts [J]. Journal of Xidian University, 2024, 51(4): 139-150.
[11]	ZHANG Xiaohong, XIANG Shijun, HUANG Hongbin. Hiding images in audio based on invertible neural networks [J]. Journal of Xidian University, 2024, 51(4): 226-238.
[12]	WAN Pengwu, HUI Xi, CHEN Dongrui, WU Bo. Modulation recognition based on the two-dimensional asynchronous in-phase quadrature histogram [J]. Journal of Xidian University, 2024, 51(4): 78-90.
[13]	HENG Hongjun, YU Longwei. Time series anomaly detection based on multi-scale feature information fusion [J]. Journal of Xidian University, 2024, 51(3): 203-214.
[14]	GONG Junyang, FU Weihong, FANG Houzhang. Research on lightweight and feature enhancement of SAR image ship targets detection [J]. Journal of Xidian University, 2024, 51(2): 96-106.
[15]	ZHONG Hao, BIAN Shan, WANG Chuntao. Real world image tampering localization combining the self-attention mechanism and convolutional neural networks [J]. Journal of Xidian University, 2024, 51(1): 135-146.