一种3D眼镜虚拟试戴系统的实现

doi:10.16180/j.cnki.issn1007-7820.2023.01.007

Abstract

Abstract:

In order to solve the problem that the mirror legs shade the face in the process of trying on 3D glasses on face image, this study proposes a virtual trying on system of 3D glasses based on face image. The three-dimensional model of the face shape is used to blank the mirror leg and solve the occlusion problem of the mirror leg. The key points of the input face image are detected, and the convex hull of the face shape is obtained by combining the Graham scanning method, and the three-dimensional model of the face shape is constructed by the translation scanning method. Additionally, according to the positioning of key points on the face image and the transformation of the 3D glasses model after pose estimation, the glasses model is worn on the face image. The experimental results show that the method achieves virtual try-on effect for multi-view face images, and solves the problem of occlusion of temples in the process of face images from multiple viewing angles. The average occlusion accuracy of the blanking leg during virtual trial is 94.5%, and the occlusion accuracy is high.

Key words: virtual try on, occlusion problem, multi-angle try-on, convex hull algorithm, Graham scanning method, key point detection, pose estimation, translation scan

CLC Number:

TP391

WANG Xiaofeng,FU Dongxiang. Realization of A Virtual Try-on System for 3D Glasses[J].Electronic Science and Technology, 2023, 36(1): 44-50.

Figures/Tables 13

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Table 1.

Table 2.

Table 3.

Figure 8.

Table 4.

Figure 9.

References 16

[1]	Corp P. Youcam makeup[EB/OL].(2017-10-18) [2021-06-16]https://www.Perfectcorp.Com/app/ymk.
[2]	胡峰松, 宋先普, 钱飞帆. 一种多视角眼镜试戴算法的研究与实现[J]. 小型微型计算机统, 2017, 38(3):605-609.
	Hu Fengsong, Song Xianpu, Qian Feifan. Research and implementation of a multi-perspective glasses try-on algorithm[J]. Journal of Chinese Computer Systems, 2017, 38(3):605-609.
[3]	Feng Z M, Jiang F, Shen R M. Virtual glasses try-on based on large pose estimation[J]. Procedia Computer Science, 2018, 131(4):226-233. doi: 10.1016/j.procs.2018.04.207
[4]	Marelli D, Bianco S, Ciocca G. A web application for glasses virtual try-on in 3D space[C]. Milano: Proceeding of the IEEE Twenty-fourth International Symposium on Consumer Technologies, 2019.
[5]	Azevedo P, Dos Santos T O, De Aguiar E. An augmented reality virtual glasses try-on system[C]. Vitoria: Proceeding of the XVIII Symposium on Virtual and Augmented Reality, 2016.
[6]	Feng Y, Wu F, Shao X, et al. Joint 3D face reconstruction and dense alignment with position map regression network[C]. Munich: Proceeding of the European Conference on Computer Vision, 2018.
[7]	刘云. 基于增强现实的虚拟眼镜试戴技术研究[D]. 大连: 大连海事大学, 2017.
	Liu Yun. Research on virtual glasses try-on technology based on augmented reality[D]. Dalian: Dalian Maritime University, 2017.
[8]	祁美晶. 基于人脸图像分析的虚拟眼镜试戴技术研究[D]. 大连: 大连海事大学, 2019.
	Qi Meijing. Research on virtual glasses try-on technology based on face image analysis[D]. Dalian: Dalian Maritime University, 2019.
[9]	Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees[C]. Columbus: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[10]	杨思燕, 苗凯彬, 王锋, 等. 视频图像中人脸自动检测与统计算法[J]. 电子科技, 2020, 33(8):1-9.
	Yang Siyan, Miao Kaibin, Wang Feng, et al. Design and implementation of automatic face detection and statistical algorithm in video images[J]. Electronic Science and Technology, 2020, 33(8):1-9.
[11]	龙沁圆, 梅钢. 基于GPU的二维凸壳计算并行Graham扫描算法(英文)[J]. 湖南师范大学自然科学学报, 2020, 43(6):66-73.
	Long Qinyuan, Mei Gang. gScan: A parallel Graham scan algorithm for calculating two-dimensional convex hulls on graphic processing units[J]. Journal of Natural Science of Hunan Normal University, 2020, 43(6):66-73.
[12]	Angel E. 交互式计算机图形学-基于OPENGL的自顶向下方法[M]. 北京: 电子工业出版社, 2009.
	Angel E. Interactive computer graphics-top-down method based on OPENGL[M]. Beijing: Press of Electronics Industry, 2009.
[13]	Li S G, Li J F. Gaze estimation from color image based on the eye model with known head pose[J]. IEEE Transactions on Human-Machine Systems, 2016, 46(3):414-423. doi: 10.1109/THMS.2015.2477507
[14]	Hsu H W, Wu T Y, Wong W H, et al. QuatNet: Quaternion-based head pose estimation with multiregression loss[J]. IEEE Transactions on Multimedia, 2018, 21(4):1035-1046. doi: 10.1109/TMM.2018.2866770
[15]	Khan A, Hayat S, Ahmad M, et al. Learning-detailed 3D face reconstruction based on convolutional neural networks from a single image[J]. Neural Computing and Applications, 2020, 33(11):5951-5964. doi: 10.1007/s00521-020-05373-w
[16]	张阵委, 章伟, 龙林, 等. 基于深度相机的小型无人机室内三维地图构建[J]. 电子科技, 2021, 34(1):65-70.
	Zhang Zhenwei, Zhang Wei, Long Lin, et al. 3D map construction of micro UAV based on depth camera[J]. Electronic Science and Technology, 2021, 34(1):65-70.

图像	全部关键点	外轮廓关键点	构建人脸形状
尺寸	凸包/ms	凸包/ms	三维模型/ms
1 280×720	2.6	0.9	21
855×480	1.6	0.6	19
455×255	1.5	0.5	18

视图	误差率	准确度
正脸图	5.5%	94.5%
左脸图	4.7%	95.3%
右脸图	6.4%	93.6%

方法	时间
PRNet	0.260 0 s/image
Deepgaze	0.060 0 s/image
本文方法	0.002 8 s/image

方法	人脸输入	眼镜输入	遮挡问题
文献[2]	图像	三维眼镜	未解决
文献[8]	视频	三维眼镜	未解决
本文方法	图像	三维眼镜	解决

Realization of A Virtual Try-on System for 3D Glasses

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 16

Related Articles 5

Metrics

Comments

Recommended 10

[1]	ZHANG Manjie,YANG Fangyan,JI Yunfeng. Research Progress of Body Posture Estimation in Ball Games [J]. Electronic Science and Technology, 2023, 36(1): 28-37.
[2]	ZHANG Xindong,FU Dongxiang. Glasses Virtual Try-on System Based on 3D Frame Modeling [J]. Electronic Science and Technology, 2022, 35(9): 52-57.
[3]	Hongjun SAN,Wanglin WANG,Jiupeng CHEN,Feiya XIE,Yangyang XU,Jia CHEN. VSLAM for Indoor Dynamic Scenes [J]. Electronic Science and Technology, 2022, 35(4): 14-19.
[4]	SU Bo,CHAI Ziqiang,WANG Li,CUI Shuaihua. Eight-Section Brocade Sequence Action Recognition and Evaluation Based on Pose Estimation [J]. Electronic Science and Technology, 2022, 35(12): 84-90.
[5]	SUN Hong,LIAO Lei. Real-time Gesture Recognition with Multiple Cues Based on OpenCV [J]. , 2015, 28(8): 145-.