基于三维镜框建模的眼镜虚拟试戴系统

doi:10.16180/j.cnki.issn1007-7820.2022.09.008

Abstract

Abstract:

In the virtual try-on system, virtual glasses need to be superimposed on the face image accurately, which determines the experience of the virtual try-on system. The key technology is to construct a real glasses model and quickly and accurately estimate the 3D face pose of the image. In response to this technical requirement, a virtual try-on technology combining 3D mirror frame modeling and head space pose estimation based on facial feature detection is proposed and a virtual try-on system is implemented. First, the method uses ERT cascade regression to locate the facial feature points of the face images. Then, the VTK visualization tool is adopted to construct a 3D mirror frame, and the processed mirror frame image is accurately fused to the face image according to the information of the face feature points and the rotation information of the head. Finally, the accuracy of the pose estimation method is tested using the AFLW data set. The results show that the algorithm has high precision and fast speed. It can still quickly and accurately realize three-dimensional and multi-angle virtual try-on under complicated conditions of large angles, many background interferences and poor light conditions, which basically meets the requirements of virtual try-on technology.

Key words: face feature points, pose estimation, Euler angle, 3D mirror frame modeling, ERT cascade regression, VTK visualization tool, blanking, images synthesis, multi-angle virtual trial

CLC Number:

TP391

ZHANG Xindong,FU Dongxiang. Glasses Virtual Try-on System Based on 3D Frame Modeling[J].Electronic Science and Technology, 2022, 35(9): 52-57.

Figures/Tables 15

Figure 1.

Figure 2.

Table 1.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Figure 9.

Table 2.

Table 3.

Figure 10.

Table 4.

Table 5.

References 20

[1]	刘云. 基于增强现实的虚拟眼镜试戴技术研究[D]. 大连: 大连海事大学, 2017.
	Liu Yun. Research on virtual glasses try-on technology based on augmented reality[D]. Dalian: Dalian Maritime University, 2017.
[2]	闫思阳. 基于人脸特征提取的技术研究与应用[D]. 北京: 北京邮电大学, 2017.
	Yan Siyang. Research and application based on face feature extraction[D]. Beijing: Beijing University of Posts and Telecommunications, 2017.
[3]	Marelli D, Bianco S, Ciocca G. A web application for glasses virtual try-on in 3D space[C]. Ancona: Proceedings of IEEE the Twenty-third International Symposium on Consumer Technologies, 2019.
[4]	祁美晶. 基于人脸图像分析的虚拟眼镜试戴技术研究[D]. 大连: 大连海事大学, 2019.
	Qi Meijing. Research on virtual glasses try on technology based on face image analysis[D]. Dalian: Dalian Maritime University, 2017.
[5]	Niswar A, Khan I R, Farbiz F. Virtual try-on of eyeglasses using 3D model of the head[C]. Hong Kong: Proceedings of the Tenth International Conference on Virtual Reality Continuum and Its Applications in Industry, 2011.
[6]	Derkach D, Ruiz A, Sukno F. 3D head pose estimation using tensor decomposition and non-linear manifold modeling[C]. Verona:Proceedings of the International Conference on 3D Vision, 2018.
[7]	Azevedo P, DosSantos T O,De Aguiar E. An augmented reality virtual glasses try-on system[C]. Gramado: Proceedings of the XVIII Symposium on Virtual and Augmented Reality, 2016.
[8]	刘芬. 基于线性和非线性算法的高效人脸识别系统[J]. 电子科技, 2019, 32(7):82-86.
	Liu Fen. An efficient face recognition system based on linear and nonlinear algorithms[J]. Electronic Science and Technology, 2019, 32(7):82-86.
[9]	司琴, 李菲菲, 陈虬. 基于深度学习与特征融合的人脸识别算法[J]. 电子科技, 2020, 33(4):18-22.
	Si Qin, Li Feifei, Chen Qiu. Face recognition algorithm based on deep learning and feature fusion[J]. Electronic Science and Technology, 2020, 33(4):18-22.
[10]	Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees[C]. Columbus: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[11]	谭小宏. 基于OpenCV&OpenGL的虚拟眼镜试戴系统的设计与实现[D]. 广州: 华南理工大学, 2017.
	Tan Xiaohong. Design and realization of virtual eyeglasses wearing system based on OpenCV and OpenGL[D]. Guangzhou: South China University of Technology, 2017.
[12]	Feng Z M, Jiang F, Shen R M. Virtual glasses try-on based on large pose estimation[J]. Procedia Computer Science, 2018, 11(7):226-233.
[13]	李明, 李晋芳, 何汉武, 等. 面向虚拟试戴系统的人脸位姿估计方法[J]. 计算机辅助设计与图形学学报, 2020, 32(12):1985-1993.
	Li Ming, Li Jinfang, He Hanwu, et al. A face pose estimation method for virtual try-on system[J]. Journal of Computer-Aided Design & Computer Graphics, 2020, 32(12):1985-1993.
[14]	Zhang Z Y. Flexible camera calibration by viewing a plane from unknown orientations[C]. Kerkyra: Proceedings of the Seventh IEEE International Conference on Computer Vision,1999.
[15]	Zhao K, Sun Q Y, Liu Z Z. 3D reconstruction of Human head CT images based on VTK[C]. Shenzhen: Proceedings of the Fifth IEEE International Conference on Advanced Robotics and Mechatronics, 2020.
[16]	Pan Z X, Tian S, Guo M Z, et al. Comparison of medical image 3D reconstruction rendering methods for robot-assisted surgery[C]. Hefei: Proceedings of the Second International Conference on Advanced Robotics and Mechatronics, 2017.
[17]	Kobayashi T, Sungiura Y, Saito H, et al. Automatic eyeglasses replacement for a 3D virtual try-on system[C]. Reims: Proceedings of the Tenth Augmented Human International Conference, 2019.
[18]	胡峰松, 宋先普, 钱飞帆. 一种多视角眼镜试戴算法的研究与实现[J]. 小型微型计算机系统, 2017, 38(3):605-609.
	Hu Fengsong, Song Xianpu, Qian Feifan. Research and implementation of a muti-perspective glasses try-on algorithm[J]. Journal of Chinese Computer Systems, 2017, 38(3):605-609.
[19]	黄亨利, 梁艳, 陈靖宇, 等. 一种基于人脸检测的虚拟眼镜试戴技术[J]. 信息通信, 2017(6):63-66.
	Huang Hengli, Liang Yan, Chen Jingyu, et al. A technology of virtual glasses try-on based on face detection[J]. Information & Communications, 2017(6):63-66.
[20]	卢洋, 王世刚, 赵文婷, 等. 基于人脸姿态估计的虚拟眼镜试戴技术[J]. 中国光学, 2015, 8(4):582-588. doi: 10.3788/co.20150804.0582
	Lu Yang, Wang Shigang, Zhao Wenting, et al. Technology of virtual eyeglasses try-on system based on face pose estimation[J]. Chinese Optics, 2015, 8(4):582-588. doi: 10.3788/co.20150804.0582

名称	X坐标	Y坐标	Z坐标
鼻尖	0.0	0.0	0.0
下巴	0.0	-330.0	-65.0
左眼角	-225.0	170.0	-135.0
右眼角	225.0	170.0	-135.0
左嘴角	-150.0	-150.0	-125.0
右嘴角	150.0	-150.0	-125.0

角度范围	俯仰	转动	摆动
±15°	8.3°	5.1°	8.7°
±30°	8.6°	7.5°	10.2°
±60°	9.1°	8.3°	11.5°

误差范围	俯仰/%	转动/%	摆动/%
±10°	71.7	75.8	65.3
±15°	85.6	88.7	75.9
±20°	92.3	92.5	85.3

方法	效果	特殊条件	消隐处理	大姿态
文献[20]	一般	不需要	不支持	不支持
文献[7]	一般	深度相机	未处理	不支持
文献[5]	良好	3D人脸	支持	不支持
本文	良好	不需要	支持	支持

Glasses Virtual Try-on System Based on 3D Frame Modeling

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 20

Related Articles 2

Metrics

Comments

Recommended 10

[1]	Hongjun SAN,Wanglin WANG,Jiupeng CHEN,Feiya XIE,Yangyang XU,Jia CHEN. VSLAM for Indoor Dynamic Scenes [J]. Electronic Science and Technology, 2022, 35(4): 14-19.
[2]	LIU Xuan,FU Dongxiang. Multi-Angle Virtual Glasses Trial Technology Based on Face Attitude Estimation [J]. Electronic Science and Technology, 2021, 34(9): 58-65.