基于人脸姿态估计的多角度虚拟眼镜试戴技术

doi:10.16180/j.cnki.issn1007-7820.2021.09.011

Abstract

Abstract:

In view of the problems that current high proportion of manual operations for virtual glasses try-on, single try-on angle, distorted try-on effect, and low commercial rate of try-on technology, a virtual try based on the combination of facial feature detection and head space pose estimation is proposed, and a virtual glasses try-on system is implemented. The real-time videos of the head are collected through the camera and analyzed by frame, and ERT cascade regression is used to locate facial feature points. According to the position information of the facial feature points and the head rotation information, the perspective-transformed eyeglass image is accurately integrated into the face image at the corresponding angle. The 3D real-time pose estimation is quantitatively evaluated on the AFLW face database, and the angle error within ±60° is about 8.8°. The results show that the algorithm has high precision and fast speed, and can quickly and accurately realize 3D multi-angle virtual try-on in complex situations with frequent angle changes, large changes in external light intensity, and many background interferences.The effect obtained by the algorithm is realistic and natural, and basically meets the requirements of virtual try-on technology.

Key words: real-time head attitude estimation, dynamic positioning, face feature point location, Euler angle, perspective transformation, image synthesis, multi-angle virtual trial, ERT, three-dimensional space

CLC Number:

TP391

LIU Xuan,FU Dongxiang. Multi-Angle Virtual Glasses Trial Technology Based on Face Attitude Estimation[J].Electronic Science and Technology, 2021, 34(9): 58-65.

Figures/Tables 17

Figure 1.

Figure 2.

Table 1

Figure 3.

Figure 4.

Table 2

Table 3

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Figure 9.

Figure 10.

Table 4

Table 5

Figure 11.

Table 6

References 20

[1]	闫思阳. 基于人脸特征提取的技术研究与应用[D]. 北京:北京邮电大学, 2017.
	Yan Siyang. Research and application based on face feature ectraction[D]. Beijing:Beijing University of Posts and Telecommunications, 2017.
[2]	张景森. 基于视觉的虚拟眼镜试戴系统开发[D]. 杭州:杭州电子科技大学, 2015.
	Zhang Jingsen. The development of virtual eye-glasses try-on system based on vision[D]. Hangzhou:Hangzhou Dianzi University, 2015.
[3]	胡畔. 基于人眼定位识别算法的眼镜试戴系统的研究与实现[D]. 长沙:湖南大学, 2016.
	Hu Pan. Research and implementation of glasses test system based on eye location recognition algorithm[D]. Changsha:Hunan University, 2016.
[4]	宿方琪. 结合手势识别的眼镜在线适配关键技术及应用平台[D]. 济南:山东大学, 2016.
	Su Fangqi. The key technology and application platform of online eyeglasses fitting based on hand gesture recognition[D]. Ji'nan:Shandong University, 2016.
[5]	杜瑶, 王兆仲. 单幅图像真实感虚拟试戴技术[J]. 计算机系统应用, 2015, 24(4):19-25.
	Du Yao, Wang Zhaozhong. Real-like virtual fitting for single image[J]. Computer Systems & Applications, 2015, 24(4):19-25.
[6]	卢洋, 王世刚, 赵文婷, 等. 基于人脸姿态估计的虚拟眼镜试戴技术[J]. 中国光学, 2015, 8(4):582-588.
	Lu Yang, Wang Shigang, Zhao Wenting, et al. Technology of virtual eyeglasses try-on system based on face pose estimation[J]. Chinese Optics, 2015, 8(4):582-588.
[7]	李鹃. 基于特征点定位的虚拟试戴的研究[D]. 上海:上海交通大学, 2011.
	Li Juan. Research on virtual try-on based on feature points extraction[D]. Shanghai:Shanghai Jiao Tong University, 2011.
[8]	刘芬. 基于线性和非线性算法的高效人脸识别系统[J]. 电子科技, 2019, 32(7):82-86.
	Liu Fen. An efficient face recognition system based on linear and nonlinear algorithms[J]. Electronic Science and Technology, 2019, 32(7):82-86.
[9]	胡少聪. 基于深度学习的人脸识别方法研究[J]. 电子科技, 2019, 32(6):82-86.
	Hu Shaocong, Research on face recognition based on deep learning[J]. Electronic Science and Technology, 2019, 32(6):82-86.
[10]	司琴, 李菲菲, 陈虬. 基于深度学习与特征融合的人脸识别算法[J]. 电子科技, 2020, 33(4):18-22.
	Si Qin, Li Feifei, Chen Qiu. Face recognition algorithm based on deep learning and feature fusion[J]. Electronic Science and Technology, 2020, 33(4):18-22.
[11]	Kazemi V, Sullivan J. One millisecond face alignment with an ensemble of regression trees[C]. Columbus:The Twenty-seventh IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[12]	Ren S Q, Cao X D, Wei Y C, et al. Face alignment at 3000 FPS via regressing local binary features[C]. Columbus:The Twenty-seventh IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[13]	Lin W H, Wang P, Tsai C F. Face recognition using support vector model classifier for user authentication[J]. Electronic Commerce Research and Applications, 2016(18):71-82.
[14]	周暖云. 人脸定位跟踪的增强现实系统研究与实现[D]. 上海:复旦大学, 2011.
	Zhou Nuanyun. Research and implementation of augmented reality system for face location and tracking[D]. Shanghai:Fudan University, 2011.
[15]	王竟磊. 基于两条垂直直线的相机标定方法研究[D]. 兰州:兰州大学, 2019.
	Wang Jinglei. Research on camera calibration method based on two vertical lines[D]. Lanzhou:Lanzhou University, 2019.
[16]	刘成. 一种基于增强现实的虚拟眼镜试戴的方法[D]. 南京:东南大学, 2015.
	Liu Cheng. A method of virtual glasses try-on based on augmented reality[D]. Nanjing:Southeast University, 2015.
[17]	刘成, 汪丰, 祁长红, 等. 一种基于增强现实的虚拟眼镜试戴的方法[J]. 工业控制计算机, 2014, 27(12):66-68.
	Liu Cheng, Wang Feng, Qi Changhong, et al. A method of virtual glasses try-on based on augmented reality[J]. Industrial Control Computer, 2014, 27(12):66-68.
[18]	陈再良. 图像感兴趣区域提取方法研究[D]. 长沙:中南大学, 2012.
	Chen Zailiang. Research on region of interest extraction[D]. Changsha:Central South University, 2012.
[19]	Koestinger M, Wohlhart P, Roth P M, et al. Annotated facial landmarks in the wild:A large-scale, real-world database for facial landmark localization[C]. Barcelona:IEEE International Conference on Computer Vision, 2012.
[20]	胡峰松, 宋先普, 钱飞帆. 一种多视角眼镜试戴算法的研究与实现[J]. 小型微型计算机系统, 2017, 38(3):605-609.
	Hu Fengsong, Song Xianpu, Qian Feifan. Research and implementation of a multi-perspective glasses try-on algorithm[J]. Journal of Chinese Computer Systems, 2017, 38(3):605-609.

特征点	X坐标	Y坐标
1	509	1 233
2	517	1 372
3	532	1 509
4	549	1 641
5	589	1 764
6	659	1 879
7	754	1 976
8	858	2 047
9	981	2 074
10	1 098	2 058

姿态变化	Pitch	Yaw	Roll
头部向下低	0°~ 90°	-	-
头部向上扬	0°~ -90°	-	-
头部向右偏转	-	0°~ 90°	-
头部向左偏转	-	0°~ -90°	-
头部向右滚动	-	-	0°~ -90°
头部向左滚动	-	-	0°~ 90°

名称	X坐标	Y坐标	Z坐标
鼻尖	0.0	0.0	0.0
下颌	0.0	-330.0	-65.0
左眼角	-225.0	170.0	-135.0
右眼角	225.0	170.0	-135.0
左嘴角	-150.0	-150.0	-125.0
右嘴角	150.0	-150.0	-125.0

角度范围	俯仰	转动	摆动
±15°	7.5°	3.8°	6.3°
±30°	8.2°	5.9°	8.7°
±60°	8.7°	7.4°	9.5°

误差范围	俯仰/%	转动/%	摆动/%
±10°	73.6	76.1	63.9
±15°	86.4	88.5	75.3
±20°	93.5	93.1	86.9

Multi-Angle Virtual Glasses Trial Technology Based on Face Attitude Estimation

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 17

References 20

Related Articles 15

Metrics

Comments

Recommended 0

[1]	FENG Zhen, MOU Haiming, XUE Jie, LI Qingdu. Omnidirectional Gait Generation Method for Biped Robot with Fusion of Imitation Learning [J]. Electronic Science and Technology, 2025, 38(1): 29-36.
[2]	GAO Yuanlong, WU Chengming, ZHANG Lei. Design and Implementation of Distribution Network Online Monitoring System Based on C# [J]. Electronic Science and Technology, 2024, 37(9): 20-26.
[3]	CHEN Jiaxing, ZHANG Xinlin, JIANG Bo, ZHANG Jiahai, HUANG Xing, QIN Yangzhen, LU Hongmin. Design and Implementation of EMC Test Antenna Based on Active Matching Technology [J]. Electronic Science and Technology, 2024, 37(9): 72-78.
[4]	WANG Yumei, ZHANG Jiqin, ZHOU Yongxin. Research on Power Grid Dispatching Considering Demand Response and Wind Power Uncertainty [J]. Electronic Science and Technology, 2024, 37(8): 68-74.
[5]	WANG Shengxiong, LIU Ruian, YAN Da. Image Style Transfer Algorithm Based on Improved Generative Adversarial Network [J]. Electronic Science and Technology, 2024, 37(6): 36-43.
[6]	WU Qiang, LIU Li, LI Shengqing, DENG Na, FENG Haotian. A Thirteen-Level Inverter Based on Switching Capacitor [J]. Electronic Science and Technology, 2024, 37(6): 51-60.
[7]	WU Jiacheng, YU Xiao. A Review of Research on Cybersecurity Risk Assessment Methods [J]. Electronic Science and Technology, 2024, 37(3): 10-17.
[8]	YUAN Qingqing, WU Ruiqi, MA Ting, XIE Xiaotong. Simplified Model Predictive Current Control for the Six-Phase Full-Bridge Inverter Fed PMSM Drive [J]. Electronic Science and Technology, 2024, 37(3): 34-43.
[9]	ZHANG Tangqian, HE Yu, JIANG Muning, QIN Tingxiang, ZHU Zhaoqiang, CHEN Zeshuang. Distributionally Robust Low-Carbon Economic Dispatch Considering New Energy Output [J]. Electronic Science and Technology, 2024, 37(10): 40-47.
[10]	ZHANG Yang, LIU Jin. Character Enhancement Based on Named Entity Recognition for Industrial Equipment Faults [J]. Electronic Science and Technology, 2024, 37(10): 48-54.
[11]	LAN Ruijie,MENG Weigao,GENG Jinqiang. Research on Real Estate Price Index Based on Sparrow Search Optimization SVR Model [J]. Electronic Science and Technology, 2024, 37(1): 1-8.
[12]	XIE Hongxing,LU Hongmin,LIU Liang,REN Yongda,LI Min,ZHANG Jiahai. Overview of Gallium Nitride Solid State Power Amplifier Development [J]. Electronic Science and Technology, 2023, 36(8): 65-71.
[13]	WEI Shenghui,HAO Zhenghang,CHEN Zhuo. Research on MMC Circulation Suppression Strategy Based on SOGI [J]. Electronic Science and Technology, 2023, 36(7): 16-23.
[14]	ZHAN Fei,YAN Yongmou. Design of A High Frequency Vehicle Small Loop Antenna with Auto-Tuning Characteristic [J]. Electronic Science and Technology, 2023, 36(6): 16-20.
[15]	ZHU Lingtong,JIANG Quan. A Novel PMSM Load Torque and Moment of Inertia Identification Strategy [J]. Electronic Science and Technology, 2023, 36(6): 80-86.