基于三维镜框建模的眼镜虚拟试戴系统

doi:10.16180/j.cnki.issn1007-7820.2022.09.008

摘要/Abstract

摘要：

在虚拟试戴系统中,虚拟眼镜需要精确地叠加到人脸图像上,这决定了虚拟试戴系统的体验感,其中的关键技术在于构建真实的眼镜模型以及快速准确地估计图像中人脸的三维空间姿态。针对该要求,文中提出了将三维镜框建模与基于人脸特征检测的头部空间姿态估计相结合的虚拟试戴技术,并实现了一个虚拟试戴系统。首先采用ERT级联回归定位人脸图像面部特征点;然后通过VTK可视化工具构建三维镜框,根据人脸特征点的信息和头部的旋转信息将处理后的镜框图像准确融合到人脸图像上;最后使用AFLW人脸数据库检验姿态估计的精确度。结果表明,该算法精度高、速度快,其在大角度、背景干扰物较多、光线条件差等复杂情况下仍能快速准确地实现三维多角度虚拟试戴,效果自然逼真,基本满足虚拟试戴技术的要求。

关键词: 人脸特征检点, 姿态估计, 欧拉角, 三维镜框建模, ERT级联回归, VTK可视化工具, 消隐处理, 图像合成, 多角度虚拟试戴

Abstract:

In the virtual try-on system, virtual glasses need to be superimposed on the face image accurately, which determines the experience of the virtual try-on system. The key technology is to construct a real glasses model and quickly and accurately estimate the 3D face pose of the image. In response to this technical requirement, a virtual try-on technology combining 3D mirror frame modeling and head space pose estimation based on facial feature detection is proposed and a virtual try-on system is implemented. First, the method uses ERT cascade regression to locate the facial feature points of the face images. Then, the VTK visualization tool is adopted to construct a 3D mirror frame, and the processed mirror frame image is accurately fused to the face image according to the information of the face feature points and the rotation information of the head. Finally, the accuracy of the pose estimation method is tested using the AFLW data set. The results show that the algorithm has high precision and fast speed. It can still quickly and accurately realize three-dimensional and multi-angle virtual try-on under complicated conditions of large angles, many background interferences and poor light conditions, which basically meets the requirements of virtual try-on technology.

Key words: face feature points, pose estimation, Euler angle, 3D mirror frame modeling, ERT cascade regression, VTK visualization tool, blanking, images synthesis, multi-angle virtual trial

中图分类号:

TP391

章锌栋,付东翔. 基于三维镜框建模的眼镜虚拟试戴系统[J]. 电子科技, 2022, 35(9): 52-57.

ZHANG Xindong,FU Dongxiang. Glasses Virtual Try-on System Based on 3D Frame Modeling[J]. Electronic Science and Technology, 2022, 35(9): 52-57.

图/表 15

图1

图2

表1

图3

图4

图5

图6

图7

图8

图9

表2

表3

图10

表4

表5

参考文献 20

[1]	刘云. 基于增强现实的虚拟眼镜试戴技术研究[D]. 大连: 大连海事大学, 2017.
	Liu Yun. Research on virtual glasses try-on technology based on augmented reality[D]. Dalian: Dalian Maritime University, 2017.
[2]	闫思阳. 基于人脸特征提取的技术研究与应用[D]. 北京: 北京邮电大学, 2017.
	Yan Siyang. Research and application based on face feature extraction[D]. Beijing: Beijing University of Posts and Telecommunications, 2017.
[3]	Marelli D, Bianco S, Ciocca G. A web application for glasses virtual try-on in 3D space[C]. Ancona: Proceedings of IEEE the Twenty-third International Symposium on Consumer Technologies, 2019.
[4]	祁美晶. 基于人脸图像分析的虚拟眼镜试戴技术研究[D]. 大连: 大连海事大学, 2019.
	Qi Meijing. Research on virtual glasses try on technology based on face image analysis[D]. Dalian: Dalian Maritime University, 2017.
[5]	Niswar A, Khan I R, Farbiz F. Virtual try-on of eyeglasses using 3D model of the head[C]. Hong Kong: Proceedings of the Tenth International Conference on Virtual Reality Continuum and Its Applications in Industry, 2011.
[6]	Derkach D, Ruiz A, Sukno F. 3D head pose estimation using tensor decomposition and non-linear manifold modeling[C]. Verona:Proceedings of the International Conference on 3D Vision, 2018.
[7]	Azevedo P, DosSantos T O,De Aguiar E. An augmented reality virtual glasses try-on system[C]. Gramado: Proceedings of the XVIII Symposium on Virtual and Augmented Reality, 2016.
[8]	刘芬. 基于线性和非线性算法的高效人脸识别系统[J]. 电子科技, 2019, 32(7):82-86.
	Liu Fen. An efficient face recognition system based on linear and nonlinear algorithms[J]. Electronic Science and Technology, 2019, 32(7):82-86.
[9]	司琴, 李菲菲, 陈虬. 基于深度学习与特征融合的人脸识别算法[J]. 电子科技, 2020, 33(4):18-22.
	Si Qin, Li Feifei, Chen Qiu. Face recognition algorithm based on deep learning and feature fusion[J]. Electronic Science and Technology, 2020, 33(4):18-22.
[10]	Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees[C]. Columbus: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[11]	谭小宏. 基于OpenCV&OpenGL的虚拟眼镜试戴系统的设计与实现[D]. 广州: 华南理工大学, 2017.
	Tan Xiaohong. Design and realization of virtual eyeglasses wearing system based on OpenCV and OpenGL[D]. Guangzhou: South China University of Technology, 2017.
[12]	Feng Z M, Jiang F, Shen R M. Virtual glasses try-on based on large pose estimation[J]. Procedia Computer Science, 2018, 11(7):226-233.
[13]	李明, 李晋芳, 何汉武, 等. 面向虚拟试戴系统的人脸位姿估计方法[J]. 计算机辅助设计与图形学学报, 2020, 32(12):1985-1993.
	Li Ming, Li Jinfang, He Hanwu, et al. A face pose estimation method for virtual try-on system[J]. Journal of Computer-Aided Design & Computer Graphics, 2020, 32(12):1985-1993.
[14]	Zhang Z Y. Flexible camera calibration by viewing a plane from unknown orientations[C]. Kerkyra: Proceedings of the Seventh IEEE International Conference on Computer Vision,1999.
[15]	Zhao K, Sun Q Y, Liu Z Z. 3D reconstruction of Human head CT images based on VTK[C]. Shenzhen: Proceedings of the Fifth IEEE International Conference on Advanced Robotics and Mechatronics, 2020.
[16]	Pan Z X, Tian S, Guo M Z, et al. Comparison of medical image 3D reconstruction rendering methods for robot-assisted surgery[C]. Hefei: Proceedings of the Second International Conference on Advanced Robotics and Mechatronics, 2017.
[17]	Kobayashi T, Sungiura Y, Saito H, et al. Automatic eyeglasses replacement for a 3D virtual try-on system[C]. Reims: Proceedings of the Tenth Augmented Human International Conference, 2019.
[18]	胡峰松, 宋先普, 钱飞帆. 一种多视角眼镜试戴算法的研究与实现[J]. 小型微型计算机系统, 2017, 38(3):605-609.
	Hu Fengsong, Song Xianpu, Qian Feifan. Research and implementation of a muti-perspective glasses try-on algorithm[J]. Journal of Chinese Computer Systems, 2017, 38(3):605-609.
[19]	黄亨利, 梁艳, 陈靖宇, 等. 一种基于人脸检测的虚拟眼镜试戴技术[J]. 信息通信, 2017(6):63-66.
	Huang Hengli, Liang Yan, Chen Jingyu, et al. A technology of virtual glasses try-on based on face detection[J]. Information & Communications, 2017(6):63-66.
[20]	卢洋, 王世刚, 赵文婷, 等. 基于人脸姿态估计的虚拟眼镜试戴技术[J]. 中国光学, 2015, 8(4):582-588. doi: 10.3788/co.20150804.0582
	Lu Yang, Wang Shigang, Zhao Wenting, et al. Technology of virtual eyeglasses try-on system based on face pose estimation[J]. Chinese Optics, 2015, 8(4):582-588. doi: 10.3788/co.20150804.0582

名称	X坐标	Y坐标	Z坐标
鼻尖	0.0	0.0	0.0
下巴	0.0	-330.0	-65.0
左眼角	-225.0	170.0	-135.0
右眼角	225.0	170.0	-135.0
左嘴角	-150.0	-150.0	-125.0
右嘴角	150.0	-150.0	-125.0

角度范围	俯仰	转动	摆动
±15°	8.3°	5.1°	8.7°
±30°	8.6°	7.5°	10.2°
±60°	9.1°	8.3°	11.5°

误差范围	俯仰/%	转动/%	摆动/%
±10°	71.7	75.8	65.3
±15°	85.6	88.7	75.9
±20°	92.3	92.5	85.3

方法	效果	特殊条件	消隐处理	大姿态
文献[20]	一般	不需要	不支持	不支持
文献[7]	一般	深度相机	未处理	不支持
文献[5]	良好	3D人脸	支持	不支持
本文	良好	不需要	支持	支持