一种3D眼镜虚拟试戴系统的实现

doi:10.16180/j.cnki.issn1007-7820.2023.01.007

摘要/Abstract

摘要：

针对人脸图像试戴3D眼镜过程中存在的镜腿遮挡人脸问题,文中提出一种基于人脸图像的3D眼镜虚拟试戴技术。利用构建的人脸形状的三维模型,使其在虚拟试戴中对镜腿起到消隐作用,解决镜腿的遮挡问题。文中对输入的人脸图像进行关键点检测,结合Graham扫描法求得人脸形状的凸多边形,利用平移扫描构建人脸形状的三维模型。此外,文中根据定位人脸图像上的关键点以及姿态估计后对三维眼镜模型的变换,将眼镜模型佩戴到人脸图像上。实验结果表明,该方法对于多视角的人脸图像实现了虚拟试戴效果,解决了多种视角下人脸图像试戴过程中镜腿的遮挡问题,虚拟试戴中镜腿遮挡平均准确率为94.5%,遮挡精度较高。

关键词: 虚拟试戴, 遮挡问题, 多视角试戴, 凸包算法, Graham扫描法, 关键点检测, 姿态估计, 平移扫描

Abstract:

In order to solve the problem that the mirror legs shade the face in the process of trying on 3D glasses on face image, this study proposes a virtual trying on system of 3D glasses based on face image. The three-dimensional model of the face shape is used to blank the mirror leg and solve the occlusion problem of the mirror leg. The key points of the input face image are detected, and the convex hull of the face shape is obtained by combining the Graham scanning method, and the three-dimensional model of the face shape is constructed by the translation scanning method. Additionally, according to the positioning of key points on the face image and the transformation of the 3D glasses model after pose estimation, the glasses model is worn on the face image. The experimental results show that the method achieves virtual try-on effect for multi-view face images, and solves the problem of occlusion of temples in the process of face images from multiple viewing angles. The average occlusion accuracy of the blanking leg during virtual trial is 94.5%, and the occlusion accuracy is high.

Key words: virtual try on, occlusion problem, multi-angle try-on, convex hull algorithm, Graham scanning method, key point detection, pose estimation, translation scan

中图分类号:

TP391

王晓锋,付东翔. 一种3D眼镜虚拟试戴系统的实现[J]. 电子科技, 2023, 36(1): 44-50.

WANG Xiaofeng,FU Dongxiang. Realization of A Virtual Try-on System for 3D Glasses[J]. Electronic Science and Technology, 2023, 36(1): 44-50.

图/表 13

图1

图2

图3

图4

图5

图6

图7

表1

表2

表3

图8

表4

图9

参考文献 16

[1]	Corp P. Youcam makeup[EB/OL].(2017-10-18) [2021-06-16]https://www.Perfectcorp.Com/app/ymk.
[2]	胡峰松, 宋先普, 钱飞帆. 一种多视角眼镜试戴算法的研究与实现[J]. 小型微型计算机统, 2017, 38(3):605-609.
	Hu Fengsong, Song Xianpu, Qian Feifan. Research and implementation of a multi-perspective glasses try-on algorithm[J]. Journal of Chinese Computer Systems, 2017, 38(3):605-609.
[3]	Feng Z M, Jiang F, Shen R M. Virtual glasses try-on based on large pose estimation[J]. Procedia Computer Science, 2018, 131(4):226-233. doi: 10.1016/j.procs.2018.04.207
[4]	Marelli D, Bianco S, Ciocca G. A web application for glasses virtual try-on in 3D space[C]. Milano: Proceeding of the IEEE Twenty-fourth International Symposium on Consumer Technologies, 2019.
[5]	Azevedo P, Dos Santos T O, De Aguiar E. An augmented reality virtual glasses try-on system[C]. Vitoria: Proceeding of the XVIII Symposium on Virtual and Augmented Reality, 2016.
[6]	Feng Y, Wu F, Shao X, et al. Joint 3D face reconstruction and dense alignment with position map regression network[C]. Munich: Proceeding of the European Conference on Computer Vision, 2018.
[7]	刘云. 基于增强现实的虚拟眼镜试戴技术研究[D]. 大连: 大连海事大学, 2017.
	Liu Yun. Research on virtual glasses try-on technology based on augmented reality[D]. Dalian: Dalian Maritime University, 2017.
[8]	祁美晶. 基于人脸图像分析的虚拟眼镜试戴技术研究[D]. 大连: 大连海事大学, 2019.
	Qi Meijing. Research on virtual glasses try-on technology based on face image analysis[D]. Dalian: Dalian Maritime University, 2019.
[9]	Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees[C]. Columbus: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[10]	杨思燕, 苗凯彬, 王锋, 等. 视频图像中人脸自动检测与统计算法[J]. 电子科技, 2020, 33(8):1-9.
	Yang Siyan, Miao Kaibin, Wang Feng, et al. Design and implementation of automatic face detection and statistical algorithm in video images[J]. Electronic Science and Technology, 2020, 33(8):1-9.
[11]	龙沁圆, 梅钢. 基于GPU的二维凸壳计算并行Graham扫描算法(英文)[J]. 湖南师范大学自然科学学报, 2020, 43(6):66-73.
	Long Qinyuan, Mei Gang. gScan: A parallel Graham scan algorithm for calculating two-dimensional convex hulls on graphic processing units[J]. Journal of Natural Science of Hunan Normal University, 2020, 43(6):66-73.
[12]	Angel E. 交互式计算机图形学-基于OPENGL的自顶向下方法[M]. 北京: 电子工业出版社, 2009.
	Angel E. Interactive computer graphics-top-down method based on OPENGL[M]. Beijing: Press of Electronics Industry, 2009.
[13]	Li S G, Li J F. Gaze estimation from color image based on the eye model with known head pose[J]. IEEE Transactions on Human-Machine Systems, 2016, 46(3):414-423. doi: 10.1109/THMS.2015.2477507
[14]	Hsu H W, Wu T Y, Wong W H, et al. QuatNet: Quaternion-based head pose estimation with multiregression loss[J]. IEEE Transactions on Multimedia, 2018, 21(4):1035-1046. doi: 10.1109/TMM.2018.2866770
[15]	Khan A, Hayat S, Ahmad M, et al. Learning-detailed 3D face reconstruction based on convolutional neural networks from a single image[J]. Neural Computing and Applications, 2020, 33(11):5951-5964. doi: 10.1007/s00521-020-05373-w
[16]	张阵委, 章伟, 龙林, 等. 基于深度相机的小型无人机室内三维地图构建[J]. 电子科技, 2021, 34(1):65-70.
	Zhang Zhenwei, Zhang Wei, Long Lin, et al. 3D map construction of micro UAV based on depth camera[J]. Electronic Science and Technology, 2021, 34(1):65-70.

图像	全部关键点	外轮廓关键点	构建人脸形状
尺寸	凸包/ms	凸包/ms	三维模型/ms
1 280×720	2.6	0.9	21
855×480	1.6	0.6	19
455×255	1.5	0.5	18

视图	误差率	准确度
正脸图	5.5%	94.5%
左脸图	4.7%	95.3%
右脸图	6.4%	93.6%

方法	时间
PRNet	0.260 0 s/image
Deepgaze	0.060 0 s/image
本文方法	0.002 8 s/image