基于人脸姿态估计的多角度虚拟眼镜试戴技术

doi:10.16180/j.cnki.issn1007-7820.2021.09.011

摘要/Abstract

摘要：

虚拟眼镜试戴手动操作占比高、试戴角度单一、试戴效果失真,所以试戴技术商用率较低。针对这些问题,文中提出了一种基于人脸特征检测与头部空间姿态估计相结合的虚拟试戴技术,并实现了一套虚拟眼镜试戴系统。通过摄像头采集头部实时视频,并按帧分析,采用ERT级联回归定位面部特征点。根据面部特征点位置信息和头部旋转信息将透视变换后的眼镜图像准确融合到对应角度的人脸图像中。在 AFLW人脸数据库上对三维实时姿态估计进行了量化评估,±60°内角度误差约为8.8°。结果表明,该算法精度高且速度快,在角度变化频繁、外界光线强弱变化大、背景干扰物较多的复杂情况下仍能快速准确地实现三维多角度虚拟试戴,效果逼真自然,基本满足虚拟试戴技术的要求。

关键词: 头部实时姿态估计, 动态定位, 人脸特征点定位, 欧拉角, 透视变换, 图像合成, 多角度虚拟试戴, ERT, 三维空间

Abstract:

In view of the problems that current high proportion of manual operations for virtual glasses try-on, single try-on angle, distorted try-on effect, and low commercial rate of try-on technology, a virtual try based on the combination of facial feature detection and head space pose estimation is proposed, and a virtual glasses try-on system is implemented. The real-time videos of the head are collected through the camera and analyzed by frame, and ERT cascade regression is used to locate facial feature points. According to the position information of the facial feature points and the head rotation information, the perspective-transformed eyeglass image is accurately integrated into the face image at the corresponding angle. The 3D real-time pose estimation is quantitatively evaluated on the AFLW face database, and the angle error within ±60° is about 8.8°. The results show that the algorithm has high precision and fast speed, and can quickly and accurately realize 3D multi-angle virtual try-on in complex situations with frequent angle changes, large changes in external light intensity, and many background interferences.The effect obtained by the algorithm is realistic and natural, and basically meets the requirements of virtual try-on technology.

Key words: real-time head attitude estimation, dynamic positioning, face feature point location, Euler angle, perspective transformation, image synthesis, multi-angle virtual trial, ERT, three-dimensional space

中图分类号:

TP391

刘轩,付东翔. 基于人脸姿态估计的多角度虚拟眼镜试戴技术[J]. 电子科技, 2021, 34(9): 58-65.

LIU Xuan,FU Dongxiang. Multi-Angle Virtual Glasses Trial Technology Based on Face Attitude Estimation[J]. Electronic Science and Technology, 2021, 34(9): 58-65.

图/表 17

图1

图2

表1

图3

图4

表2

表3

图5

图6

图7

图8

图9

图10

表4

表5

图11

表6

参考文献 20

[1]	闫思阳. 基于人脸特征提取的技术研究与应用[D]. 北京:北京邮电大学, 2017.
	Yan Siyang. Research and application based on face feature ectraction[D]. Beijing:Beijing University of Posts and Telecommunications, 2017.
[2]	张景森. 基于视觉的虚拟眼镜试戴系统开发[D]. 杭州:杭州电子科技大学, 2015.
	Zhang Jingsen. The development of virtual eye-glasses try-on system based on vision[D]. Hangzhou:Hangzhou Dianzi University, 2015.
[3]	胡畔. 基于人眼定位识别算法的眼镜试戴系统的研究与实现[D]. 长沙:湖南大学, 2016.
	Hu Pan. Research and implementation of glasses test system based on eye location recognition algorithm[D]. Changsha:Hunan University, 2016.
[4]	宿方琪. 结合手势识别的眼镜在线适配关键技术及应用平台[D]. 济南:山东大学, 2016.
	Su Fangqi. The key technology and application platform of online eyeglasses fitting based on hand gesture recognition[D]. Ji'nan:Shandong University, 2016.
[5]	杜瑶, 王兆仲. 单幅图像真实感虚拟试戴技术[J]. 计算机系统应用, 2015, 24(4):19-25.
	Du Yao, Wang Zhaozhong. Real-like virtual fitting for single image[J]. Computer Systems & Applications, 2015, 24(4):19-25.
[6]	卢洋, 王世刚, 赵文婷, 等. 基于人脸姿态估计的虚拟眼镜试戴技术[J]. 中国光学, 2015, 8(4):582-588.
	Lu Yang, Wang Shigang, Zhao Wenting, et al. Technology of virtual eyeglasses try-on system based on face pose estimation[J]. Chinese Optics, 2015, 8(4):582-588.
[7]	李鹃. 基于特征点定位的虚拟试戴的研究[D]. 上海:上海交通大学, 2011.
	Li Juan. Research on virtual try-on based on feature points extraction[D]. Shanghai:Shanghai Jiao Tong University, 2011.
[8]	刘芬. 基于线性和非线性算法的高效人脸识别系统[J]. 电子科技, 2019, 32(7):82-86.
	Liu Fen. An efficient face recognition system based on linear and nonlinear algorithms[J]. Electronic Science and Technology, 2019, 32(7):82-86.
[9]	胡少聪. 基于深度学习的人脸识别方法研究[J]. 电子科技, 2019, 32(6):82-86.
	Hu Shaocong, Research on face recognition based on deep learning[J]. Electronic Science and Technology, 2019, 32(6):82-86.
[10]	司琴, 李菲菲, 陈虬. 基于深度学习与特征融合的人脸识别算法[J]. 电子科技, 2020, 33(4):18-22.
	Si Qin, Li Feifei, Chen Qiu. Face recognition algorithm based on deep learning and feature fusion[J]. Electronic Science and Technology, 2020, 33(4):18-22.
[11]	Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees[C]. Columbus:The Twenty-seventh IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[12]	Ren S Q, Cao X D, Wei Y C, et al. Face alignment at 3000 FPS via regressing local binary features[C]. Columbus:The Twenty-seventh IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[13]	Lin W H, Wang P, Tsai C F. Face recognition using support vector model classifier for user authentication[J]. Electronic Commerce Research and Applications, 2016(18):71-82.
[14]	周暖云. 人脸定位跟踪的增强现实系统研究与实现[D]. 上海:复旦大学, 2011.
	Zhou Nuanyun. Research and implementation of augmented reality system for face location and tracking[D]. Shanghai:Fudan University, 2011.
[15]	王竟磊. 基于两条垂直直线的相机标定方法研究[D]. 兰州:兰州大学, 2019.
	Wang Jinglei. Research on camera calibration method based on two vertical lines[D]. Lanzhou:Lanzhou University, 2019.
[16]	刘成. 一种基于增强现实的虚拟眼镜试戴的方法[D]. 南京:东南大学, 2015.
	Liu Cheng. A method of virtual glasses try-on based on augmented reality[D]. Nanjing:Southeast University, 2015.
[17]	刘成, 汪丰, 祁长红, 等. 一种基于增强现实的虚拟眼镜试戴的方法[J]. 工业控制计算机, 2014, 27(12):66-68.
	Liu Cheng, Wang Feng, Qi Changhong, et al. A method of virtual glasses try-on based on augmented reality[J]. Industrial Control Computer, 2014, 27(12):66-68.
[18]	陈再良. 图像感兴趣区域提取方法研究[D]. 长沙:中南大学, 2012.
	Chen Zailiang. Research on region of interest extraction[D]. Changsha:Central South University, 2012.
[19]	Koestinger M, Wohlhart P, Roth P M, et al. Annotated facial landmarks in the wild:A large-scale, real-world database for facial landmark localization[C]. Barcelona:IEEE International Conference on Computer Vision, 2012.
[20]	胡峰松, 宋先普, 钱飞帆. 一种多视角眼镜试戴算法的研究与实现[J]. 小型微型计算机系统, 2017, 38(3):605-609.
	Hu Fengsong, Song Xianpu, Qian Feifan. Research and implementation of a multi-perspective glasses try-on algorithm[J]. Journal of Chinese Computer Systems, 2017, 38(3):605-609.

特征点	X坐标	Y坐标
1	509	1 233
2	517	1 372
3	532	1 509
4	549	1 641
5	589	1 764
6	659	1 879
7	754	1 976
8	858	2 047
9	981	2 074
10	1 098	2 058

姿态变化	Pitch	Yaw	Roll
头部向下低	0°~ 90°	-	-
头部向上扬	0°~ -90°	-	-
头部向右偏转	-	0°~ 90°	-
头部向左偏转	-	0°~ -90°	-
头部向右滚动	-	-	0°~ -90°
头部向左滚动	-	-	0°~ 90°

名称	X坐标	Y坐标	Z坐标
鼻尖	0.0	0.0	0.0
下颌	0.0	-330.0	-65.0
左眼角	-225.0	170.0	-135.0
右眼角	225.0	170.0	-135.0
左嘴角	-150.0	-150.0	-125.0
右嘴角	150.0	-150.0	-125.0

角度范围	俯仰	转动	摆动
±15°	7.5°	3.8°	6.3°
±30°	8.2°	5.9°	8.7°
±60°	8.7°	7.4°	9.5°

误差范围	俯仰/%	转动/%	摆动/%
±10°	73.6	76.1	63.9
±15°	86.4	88.5	75.3
±20°	93.5	93.1	86.9