基于人脸姿态估计的多角度虚拟眼镜试戴技术

doi:10.16180/j.cnki.issn1007-7820.2021.09.011

Abstract

Abstract:

In view of the problems that current high proportion of manual operations for virtual glasses try-on, single try-on angle, distorted try-on effect, and low commercial rate of try-on technology, a virtual try based on the combination of facial feature detection and head space pose estimation is proposed, and a virtual glasses try-on system is implemented. The real-time videos of the head are collected through the camera and analyzed by frame, and ERT cascade regression is used to locate facial feature points. According to the position information of the facial feature points and the head rotation information, the perspective-transformed eyeglass image is accurately integrated into the face image at the corresponding angle. The 3D real-time pose estimation is quantitatively evaluated on the AFLW face database, and the angle error within ±60° is about 8.8°. The results show that the algorithm has high precision and fast speed, and can quickly and accurately realize 3D multi-angle virtual try-on in complex situations with frequent angle changes, large changes in external light intensity, and many background interferences.The effect obtained by the algorithm is realistic and natural, and basically meets the requirements of virtual try-on technology.

Key words: real-time head attitude estimation, dynamic positioning, face feature point location, Euler angle, perspective transformation, image synthesis, multi-angle virtual trial, ERT, three-dimensional space

CLC Number:

TP391

LIU Xuan,FU Dongxiang. Multi-Angle Virtual Glasses Trial Technology Based on Face Attitude Estimation[J].Electronic Science and Technology, 2021, 34(9): 58-65.

Figures/Tables 17

Figure 1.

Figure 2.

Table 1

Figure 3.

Figure 4.

Table 2

Table 3

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Figure 9.

Figure 10.

Table 4

Table 5

Figure 11.

Table 6

References 20

[1]	闫思阳. 基于人脸特征提取的技术研究与应用[D]. 北京:北京邮电大学, 2017.
	Yan Siyang. Research and application based on face feature ectraction[D]. Beijing:Beijing University of Posts and Telecommunications, 2017.
[2]	张景森. 基于视觉的虚拟眼镜试戴系统开发[D]. 杭州:杭州电子科技大学, 2015.
	Zhang Jingsen. The development of virtual eye-glasses try-on system based on vision[D]. Hangzhou:Hangzhou Dianzi University, 2015.
[3]	胡畔. 基于人眼定位识别算法的眼镜试戴系统的研究与实现[D]. 长沙:湖南大学, 2016.
	Hu Pan. Research and implementation of glasses test system based on eye location recognition algorithm[D]. Changsha:Hunan University, 2016.
[4]	宿方琪. 结合手势识别的眼镜在线适配关键技术及应用平台[D]. 济南:山东大学, 2016.
	Su Fangqi. The key technology and application platform of online eyeglasses fitting based on hand gesture recognition[D]. Ji'nan:Shandong University, 2016.
[5]	杜瑶, 王兆仲. 单幅图像真实感虚拟试戴技术[J]. 计算机系统应用, 2015, 24(4):19-25.
	Du Yao, Wang Zhaozhong. Real-like virtual fitting for single image[J]. Computer Systems & Applications, 2015, 24(4):19-25.
[6]	卢洋, 王世刚, 赵文婷, 等. 基于人脸姿态估计的虚拟眼镜试戴技术[J]. 中国光学, 2015, 8(4):582-588.
	Lu Yang, Wang Shigang, Zhao Wenting, et al. Technology of virtual eyeglasses try-on system based on face pose estimation[J]. Chinese Optics, 2015, 8(4):582-588.
[7]	李鹃. 基于特征点定位的虚拟试戴的研究[D]. 上海:上海交通大学, 2011.
	Li Juan. Research on virtual try-on based on feature points extraction[D]. Shanghai:Shanghai Jiao Tong University, 2011.
[8]	刘芬. 基于线性和非线性算法的高效人脸识别系统[J]. 电子科技, 2019, 32(7):82-86.
	Liu Fen. An efficient face recognition system based on linear and nonlinear algorithms[J]. Electronic Science and Technology, 2019, 32(7):82-86.
[9]	胡少聪. 基于深度学习的人脸识别方法研究[J]. 电子科技, 2019, 32(6):82-86.
	Hu Shaocong, Research on face recognition based on deep learning[J]. Electronic Science and Technology, 2019, 32(6):82-86.
[10]	司琴, 李菲菲, 陈虬. 基于深度学习与特征融合的人脸识别算法[J]. 电子科技, 2020, 33(4):18-22.
	Si Qin, Li Feifei, Chen Qiu. Face recognition algorithm based on deep learning and feature fusion[J]. Electronic Science and Technology, 2020, 33(4):18-22.
[11]	Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees[C]. Columbus:The Twenty-seventh IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[12]	Ren S Q, Cao X D, Wei Y C, et al. Face alignment at 3000 FPS via regressing local binary features[C]. Columbus:The Twenty-seventh IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[13]	Lin W H, Wang P, Tsai C F. Face recognition using support vector model classifier for user authentication[J]. Electronic Commerce Research and Applications, 2016(18):71-82.
[14]	周暖云. 人脸定位跟踪的增强现实系统研究与实现[D]. 上海:复旦大学, 2011.
	Zhou Nuanyun. Research and implementation of augmented reality system for face location and tracking[D]. Shanghai:Fudan University, 2011.
[15]	王竟磊. 基于两条垂直直线的相机标定方法研究[D]. 兰州:兰州大学, 2019.
	Wang Jinglei. Research on camera calibration method based on two vertical lines[D]. Lanzhou:Lanzhou University, 2019.
[16]	刘成. 一种基于增强现实的虚拟眼镜试戴的方法[D]. 南京:东南大学, 2015.
	Liu Cheng. A method of virtual glasses try-on based on augmented reality[D]. Nanjing:Southeast University, 2015.
[17]	刘成, 汪丰, 祁长红, 等. 一种基于增强现实的虚拟眼镜试戴的方法[J]. 工业控制计算机, 2014, 27(12):66-68.
	Liu Cheng, Wang Feng, Qi Changhong, et al. A method of virtual glasses try-on based on augmented reality[J]. Industrial Control Computer, 2014, 27(12):66-68.
[18]	陈再良. 图像感兴趣区域提取方法研究[D]. 长沙:中南大学, 2012.
	Chen Zailiang. Research on region of interest extraction[D]. Changsha:Central South University, 2012.
[19]	Koestinger M, Wohlhart P, Roth P M, et al. Annotated facial landmarks in the wild:A large-scale, real-world database for facial landmark localization[C]. Barcelona:IEEE International Conference on Computer Vision, 2012.
[20]	胡峰松, 宋先普, 钱飞帆. 一种多视角眼镜试戴算法的研究与实现[J]. 小型微型计算机系统, 2017, 38(3):605-609.
	Hu Fengsong, Song Xianpu, Qian Feifan. Research and implementation of a multi-perspective glasses try-on algorithm[J]. Journal of Chinese Computer Systems, 2017, 38(3):605-609.

特征点	X坐标	Y坐标
1	509	1 233
2	517	1 372
3	532	1 509
4	549	1 641
5	589	1 764
6	659	1 879
7	754	1 976
8	858	2 047
9	981	2 074
10	1 098	2 058

姿态变化	Pitch	Yaw	Roll
头部向下低	0°~ 90°	-	-
头部向上扬	0°~ -90°	-	-
头部向右偏转	-	0°~ 90°	-
头部向左偏转	-	0°~ -90°	-
头部向右滚动	-	-	0°~ -90°
头部向左滚动	-	-	0°~ 90°

名称	X坐标	Y坐标	Z坐标
鼻尖	0.0	0.0	0.0
下颌	0.0	-330.0	-65.0
左眼角	-225.0	170.0	-135.0
右眼角	225.0	170.0	-135.0
左嘴角	-150.0	-150.0	-125.0
右嘴角	150.0	-150.0	-125.0

角度范围	俯仰	转动	摆动
±15°	7.5°	3.8°	6.3°
±30°	8.2°	5.9°	8.7°
±60°	8.7°	7.4°	9.5°

误差范围	俯仰/%	转动/%	摆动/%
±10°	73.6	76.1	63.9
±15°	86.4	88.5	75.3
±20°	93.5	93.1	86.9

Multi-Angle Virtual Glasses Trial Technology Based on Face Attitude Estimation

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 17

References 20

Related Articles 15

Metrics

Comments

Recommended 0

[1]	LIU Jianlong,HAO Zhenghang. Comparative Study of Wind Power System Simulation Based on Back-to-Back Converters [J]. Electronic Science and Technology, 2022, 35(2): 67-73.
[2]	CHEN Jiaming,WANG Lihua,SU Xiaohang. Research on Mechanical Characteristics of Railway Bridge Ballast Bed Based on ADAMS [J]. Electronic Science and Technology, 2022, 35(1): 53-59.
[3]	WANG Yumei,ZHANG Zihan,WANG Hao. Large-Disturbance Stability Analysis of DC Microgrid Based on Mixed Potential Function [J]. Electronic Science and Technology, 2022, 35(1): 66-72.
[4]	LI Yudong,LIAN Haishan,HU Xiaodan. AC/AC Frequency Conversion Power Supply Based on Three-Phase Input [J]. Electronic Science and Technology, 2021, 34(9): 47-53.
[5]	ZHANG Jie,ZENG Guohui,HUANG Bo. The Loss Calculation and Analysis of Hybrid Modular Multilevel Converter Considering Sub-Module Balancing Constraint [J]. Electronic Science and Technology, 2021, 34(8): 71-78.
[6]	YAN Zhenbin,ZHENG Bochao,ZHOU Zhiyong. Research on Grid Connection of Photovoltaic Power Generation Systems Based on a New Robust Droop Control Method [J]. Electronic Science and Technology, 2021, 34(8): 79-86.
[7]	YU Jiawei,HU Haiyang. Energy Consumption Model of Cyber-Physical System Based on Object-Oriented Fuzzy Petri Net [J]. Electronic Science and Technology, 2021, 34(7): 19-25.
[8]	CHEN Penggang,FENG Xiaoyi. Animal Disease Diagnosis Algorithm Based on Covering Rough Vague Soft Expert Set [J]. Electronic Science and Technology, 2021, 34(6): 1-10.
[9]	ZHANG Jiapeng,LI Lin,ZHU Ye. A Review of Research on Decision-Making Method of Autonomous Vehicle Based on Reinforcement Learning [J]. Electronic Science and Technology, 2021, 34(5): 66-71.
[10]	PU Jianghua,WANG Xuejun,WU Peng,CHEN Mingfang,YANG Xiong. Stability Analysis of Transmission Structure of Rail Transport Conveyor Lifting Subsystem Based on Contact [J]. Electronic Science and Technology, 2021, 34(2): 45-51.
[11]	TAO Jiaqing,HU Yue,XIANG Huazhong,TU Jiankun,ZHENG Gang. A Method of Measuring Optical Fibers’ Mode Field Diameter Based on Two-Segments Polynomial Function Fitting [J]. Electronic Science and Technology, 2021, 34(11): 37-41.
[12]	WANG Yumei,LI Yaowei. Research on Flexible Switching Controller Based on UPFC [J]. Electronic Science and Technology, 2021, 34(11): 55-61.
[13]	LI Lin,XIE Ming,HUANG Song. Design of Modular Parallel Current Sharing Single-Phase Inverter [J]. Electronic Science and Technology, 2021, 34(10): 27-31.
[14]	HUANG Yuze,CHEN Guangdong,WANG Yuan,WAN Siyu. A Common Illumination Source Super Spatial Resolution Three-dimensional Reconstruction Method [J]. Electronic Science and Technology, 2020, 33(7): 57-62.
[15]	WU Hanhan,BU Gang. Dual Master-Slave Output Strategy Based on Switching Capacitor DC-DC Converter [J]. Electronic Science and Technology, 2020, 33(6): 35-39.