结合形变模型的人体姿态估计优化算法

doi:10.19665/j.issn1001-2400.2020.02.004

Abstract

Abstract:

An optimization algorithm is proposed utilizing the video data and point cloud data captured by the depth camera to solve the problems such as error-proneness and incoherence of motion sequence caused by the existing human pose estimation algorithms based on the morphable model. For video data, the neural network is first used in extracting the model parameters from each color image frame. Next, the human key-points and contour constraint are considered to optimize the above parameters. Then the coherence between every two consecutive frames is utilized to correct the error of pose estimation, thus making the resulting motion sequence smoother. In addition, the point cloud and the model obtained from the corresponding color image frame are used as the joint input to further improve the estimation accuracy. Finally, the distance between the point cloud and the corresponding point of the model is constrained to be as small as possible to obtain a more reasonable solution. The proposed algorithm and the state-of-the-art algorithms are compared qualitatively and quantitatively on the data set and real video set. Experimental results show that the algorithm can effectively correct the error and incoherence in the single-frame pose estimation results and greatly improve the accuracy when using point cloud data optimization.

Key words: pose estimation, motion reconstruction, morphable model, point cloud, neural network

CLC Number:

TP391

LI Jian,ZHANG Haoruo,HE Bin. Optimization algorithm for estimating the human pose by using the morphable model[J].Journal of Xidian University, 2020, 47(2): 23-31.

Figures/Tables 9

References 19

[1]	WEI S E, RAMAKRISHNA V, KANADE T , et al. Convolutional Pose Machines[C]// Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2016: 4724-4732.
[2]	FANG H S, XIE S, TAI Y W , et al. Rmpe: Regional Multi-person Pose Estimation[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017: 2353-2362.
[3]	CAO Z, SIMON T, WEI S E , et al. Realtime Multi-person 2d Pose Estimation Using Part Affinity Fields[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017: 1302-1310.
[4]	LUO Y, REN J, WANG Z , et al. LSTM Pose Machines[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2018: 5207-5215.
[5]	LOPER M, MAHMOOD N, ROMERO J , et al. SMPL:a Skinned Multi-person Linear Model[J]. ACM Transactions on Graphics, 2015,34(6):248.
[6]	BOGO F, KANAZAWA A, LASSNER C , et al. Keep It SMPL:Automatic Estimation of 3D Human Pose and Shape from a Single Image[C]// Lecture Notes in Computer Science: 9909. Heidelberg:Springer Verlag, 2016: 561-578.
[7]	LASSNER C, ROMERO J, KIEFEL M , et al. Unite the People:Closing the Loop between 3d and 2d Human RepresEntations[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017: 4704-4713.
[8]	HUANG Y, BOGO F, LASSNER C , et al. Towards Accurate Marker-less Human Shape and Pose Estimation over Time[C]// Proceedings of the 2017 International Conference on 3D Vision. Piscataway:IEEE, 2018: 421-430.
[9]	ALLDIECK T, MAGNOR M, XU W , et al. Video Based Reconstruction of 3d People Models[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2018: 8387-8397.
[10]	OMRAN M, LASSNER C, PONS-MOLL G , et al. Neural Body Fitting:Unifying Deep Learning and Model Based Human Pose and Shape Estimation[C]// Proceedings of the 2018 International Conference on 3D Vision. Piscataway:IEEE, 2018: 484-494.
[11]	KANAZAWA A, BLACK M J, JACOBS D W , et al. End-to-end Recovery of Human Shape and Pose[C]// Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2018: 7122-7131.
[12]	KOLOTOUROS N, PAVLAKOS G, DANIILIDIS K . Convolutional Mesh Regression for Single-image Human Shape Reconstruction[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019: 4501-4510.
[13]	KANAZAWA A, ZHANG J, FELSEN P , et al. Learning 3D Human Dynamics from Video[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019: 5614-5623.
[14]	CAELLES S, MANINIS K K, PONT-TUSET J , et al. One-shot Video Object Segmentation[C]// Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017: 5320-5329.
[15]	LOPER M M, BLACK M J . OpenDR:an Approximate Differentiable Renderer[C]// Lecture Notes in Computer Science: 8695. Heidelberg:Springer Verlag, 2014: 154-169.
[16]	AHOOKHOSH M, AMINI K, KIMIAEI M . A Globally Convergent Trust-region Method for Large-scale Symmetric Nonlinear Systems[J]. Numerical Functional Analysis and Optimization, 2015,36(7):830-855.
[17]	IONESCU C, PAPAVA D, OLARU V , et al. Human3.6M:Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014,36(6):1325-1339.
[18]	彭亚丽, 刘侍刚, 梁新刚 , 等. 三维变形体的因式分解重建方法[J]. 西安电子科技大学学报, 2016,43(1):116-119.
	PENG Yali, LIU Shigang, LIANG Xingang , et al. Factorization Reconstruction Method for the 3D Deformable Structure[J] . Journal of Xidian University, 2016,43(1):116-119.
[19]	PAVLAKOS G, CHOUTAS V, GHORBANI N , et al. Expressive Body Capture:3D Hands, Face, and Body from a Single Image[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019: 10975-10985.

算法	PA-MPJPE/mm
文献[6]	82.3
文献[8]	58.4
文献[10]	59.9
人体网格重构算法^[11]	56.8
文献[13]	58.1
文中算法	56.3

数据	人体网格重构算法^[11]	均方根文献[9]	点云姿态估计算法
图7第一列	0.020221	0.018260	0.011648
图7第二列	0.019626	0.025794	0.013060
图7第三列	0.026306	0.026720	0.010320
图7第四列	0.028419	0.026597	0.013047
图7第五列	0.020325	0.025399	0.014320
图7第六列	0.027542	0.033031	0.012443
平均	0.023739	0.025096	0.012473

Optimization algorithm for estimating the human pose by using the morphable model

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 19

Related Articles 15

Metrics

Comments

Recommended 10

[1]	LV Wenkai,YANG Pengfei,DING Yunqing,ZHANG Heyu,ZHENG Tianyang. JEDERL:A task scheduling optimization algorithm for heterogeneous computing platforms [J]. Journal of Xidian University, 2021, 48(6): 67-74.
[2]	YU Haoyang,YIN Liang,LI Shufang,LV Shun. Recognition algorithm for the little sample radar modulation signal based on the generative adversarial network [J]. Journal of Xidian University, 2021, 48(6): 96-104.
[3]	HU Daiwang,JIAO Yiyuan,LI Yanni. Novel and efficient algorithm for entity relation extraction with the corpus knowledge graph [J]. Journal of Xidian University, 2021, 48(6): 75-83.
[4]	SUN Yanjing,WEI Li,ZHANG Nianlong,YUN Xiao,DONG Kaiwen,GE Min,CHENG Xiaozhou,HOU Xiaofeng. Person re-identification method combining the DD-GAN and Global feature in a coal mine [J]. Journal of Xidian University, 2021, 48(5): 201-211.
[5]	ZHOU Peng,YANG Jun. Semantic segmentation of remote sensing images based on neural architecture search [J]. Journal of Xidian University, 2021, 48(5): 47-57.
[6]	ZHANG Shuwei,LI Junmin. Human body detection algorithm in complex monitoring scenes [J]. Journal of Xidian University, 2021, 48(5): 68-77.
[7]	YANG Yunhang,MIN Lianquan. Multi-scalefusion sketch recognition model by dilated convolution [J]. Journal of Xidian University, 2021, 48(5): 92-99.
[8]	ZHANG Yuhao,CHENG Peitao,ZHANG Shuhao,WANG Xiumei. Lightweight image super-resolution with the adaptive weight learning network [J]. Journal of Xidian University, 2021, 48(5): 15-22.
[9]	HAN Yongsai,MA Shiping,HE Linyuan,LI Chenghao,ZHU Mingming,ZHANG Fei. Detection of the object in the fast remote sensing airport area on the improved YOLOv3 [J]. Journal of Xidian University, 2021, 48(5): 156-166.
[10]	DONG Ruchan,JIAO Licheng,ZHAO Jin,SHEN Weiyan. Application of the deep fusion mechanism in object detection of remote sensing images [J]. Journal of Xidian University, 2021, 48(5): 128-138.
[11]	CHENG De,HAO Yi,ZHOU Jingyu,WANG Nannan,GAO Xinbo. Cross-modality person re-identification utilizing the hybrid two-stream neural networks [J]. Journal of Xidian University, 2021, 48(5): 190-200.
[12]	CHEN Changchuan,WANG Haining,HUANG Lian,HUANG Tao,LI Lianjie,HUANG Xiangkang,DAI Shaosheng. Facial expression recognition based on local representation [J]. Journal of Xidian University, 2021, 48(5): 100-109.
[13]	SONG Jianfeng,MIAO Qiguang,WANG Chongxiao,XU Hao,YANG Jin. Multi-scale single object tracking based on the attention mechanism [J]. Journal of Xidian University, 2021, 48(5): 110-116.
[14]	HUI Haisheng,ZHANG Xueying,WU Zelin,LI Fenglian. Method for stroke lesion segmentation using the primary-auxiliary path attention compensation network [J]. Journal of Xidian University, 2021, 48(4): 200-208.
[15]	SONG Jianqiao,WANG Feng,NIU Jin,SHI Zezhou,MA Junhui. Potential emotion recognition based on the fusion of the spatio-temporal neural network and facial pulse signals [J]. Journal of Xidian University, 2021, 48(4): 159-167.