基于MobileNetV2与LBP特征融合的婴幼儿表情识别算法

doi:10.16180/j.cnki.issn1007-7820.2022.08.008

Abstract

Abstract:

In view of the problems of low rate of infant expression recognition and the insufficient extraction of complex features, an infant expression recognition algorithm based on MobileNetV2 and LBP dual-channel feature fusion is proposed. The first channel uses the improved MobileNetV2 network to quickly and accurately extract the global features of facial expressions. The second channel divides the original input image into blocks, and uses image information entropy to construct weights, and extracts block-weighted LBP histogram features to highlight the regions with rich expression information. The output vector of the dual-channel model is fused to improve the feature expression ability, and the support vector machine is used to replace the Softmax layer for expression classification. Experiments show that the use of fusion features has a better classification effect than a single feature, and in the self-built infant expression data set, the accuracy of expression recognition can reach 85.71%.

Key words: convolutional neural network, local binary pattern, feature fusion, dual-channel model, expression recognition, image information entropy, infant, support vector machine

CLC Number:

TP391

DENG Yuan,SHI Yiping,JIANG Yueying,ZHU Yamei,LIU Jin. Infant Expression Recognition Algorithm Based on MobileNetV2 and LBP Feature Fusion[J].Electronic Science and Technology, 2022, 35(8): 47-52.

Figures/Tables 17

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Table 1.

Figure 6.

Figure 7.

Figure 8.

Figure 9.

Figure 10.

Table 2.

Figure 11.

Figure 12.

Figure 13.

Table 3.

Table 4.

References 17

[1]	卢官明, 杨成, 杨文娟, 等. 基于LBP-TOP特征的微表情识别[J]. 南京邮电大学学报(自然科学版), 2017, 37(6):1-7.
	Lu Guanming, Yang Cheng, Yang Wenjuan, et al. Micro-expression recognition based on LBP-TOP features[J]. Journal of Nanjing University of Posts and Telecommunications(Natural Science Edition), 2017, 37(6):1-7.
[2]	胡敏, 滕文娣, 王晓华, 等. 融合局部纹理和形状特征的人脸表情识别[J]. 电子与信息学报, 2018, 40(6):1338-1344.
	Hu Min, Teng Wendi, Wang Xiaohua, et al. Facial expression recognition based on local texture and shape features[J]. Journal of Electronics & Information Technology, 2018, 40(6):1338-1344.
[3]	Sun Y, Yu J. Facial expression recognition by fusing Gabor and local binary pattern features[C]. Reykjavik:Proceedings of the International Conference on MultiMedia Modeling, 2017.
[4]	李勇, 林小竹, 蒋梦莹. 基于跨连接LeNet-5网络的面部表情识别[J]. 自动化学报, 2018, 44(1):176-182.
	Li Yong, Lin Xiaozhu, Jiang Mengying. Facial expression recognition with cross-connect LeNet-5 network[J]. Acta Automatica Sinica, 2018, 44(1):176-182.
[5]	薛建明, 刘宏哲, 袁家政, 等. 基于CNN与关键区域特征的人脸表情识别算法[J]. 传感器与微系统, 2019, 38(10):146-149.
	Xue Jianming, Liu Hongzhe, Yuan Jiazheng, et al. Facial expression recognition algorithm based on CNN and features of key regions[J]. Transducer and Microsystem Technologies, 2019, 38(10):146-149.
[6]	丁名都, 李琳. 基于CNN和HOG双路特征融合的人脸表情识别[J]. 信息与控制, 2020, 49(1):47-54. doi: 10.13976/j.cnki.xk.2020.9185
	Ding Mingdu, Li Lin. CNN and HOG dual-path feature fusion for face expression recognition[J]. Information and Control, 2020, 49(1):47-54. doi: 10.13976/j.cnki.xk.2020.9185
[7]	司琴, 李菲菲, 陈虬. 基于深度学习与特征融合的人脸识别算法[J]. 电子科技, 2020, 33(4):18-22.
	Si Qin, Li Feifei, Chen Qiu. Face recognition algorithm based on deep learning and feature fusion[J]. Electronic Science and Technology, 2020, 33(4):18-22.
[8]	何志超, 赵龙章, 陈闯. 用于人脸表情识别的多分辨率特征融合卷积神经网络[J]. 激光与光电子学进展, 2018, 55(7):370-375.
	He Zhichao, Zhao Longzhang, Chen Chuang. Convolution neural network with multi-resolution feature fusion for facial expression recognition[J]. Laser & Optoelectronics Progress, 2018, 55(7):370-375.
[9]	Sandler M, Howard A, Zhu M, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]. Piscataway:Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2018.
[10]	李嘉頔, 陈振学, 刘成云. 分块CS-LBP和加权PCA的低分辨率人脸识别[J]. 光电子·激光, 2016, 27(2):210-216.
	Li Jiadi, Chen Zhenxue, Liu Chengyun. Low-resolution face recognition based on blocking CS-LBP and weighted PCA algorithm[J]. Journal of Optoelectronics·Laser, 2016, 27(2):210-216.
[11]	陈酉明, 宋晓宁, 於东军. 基于分块加权LBP向量和解析字典的协同表达分类[J]. 南京理工大学学报, 2019, 43(2):170-176.
	Chen Youming, Song Xiaoning, Yu Dongjun. Collaborative representation based classification based on weighted block-based LBP histogram vector and analytic dictionary[J]. Journal of Nanjing University of Science and Technology, 2019, 43(2):170-176.
[12]	张翔, 史志才, 陈良. 基于SWA优化级联网络的表情识别方法[J]. 电子科技, 2020, 33(9):16-20.
	Zhang Xiang, Shi Zhicai, Chen Liang. Expression recognition method based on cascade network optimized by SWA[J]. Electronic Science and Technology, 2020, 33(9):16-20.
[13]	苏饶, 李菲菲, 陈虬. 基于多特征融合的人脸识别算法[J]. 电子科技, 2019, 32(7):43-48.
	Su Rao, Li Feifei, Chen Qiu. Face recognition algorithm based on multiple feature fusion[J]. Electronic Science and Technology, 2019, 32(7):43-48.
[14]	胡敏, 李堃, 王晓华, 等. 基于直方图加权HCBP的人脸表情识别[J]. 电子测量与仪器学报, 2015, 29(7):953-960.
	Hu Min, Li Kun, Wang Xiaohua, et al. Facial expression recognition based on histogram weighted HCBP[J]. Journal of Electronic Measurement and Instrumentation, 2015, 29(7):953-960.
[15]	Kim J H, Kim B G, Roy P P, et al. Efficient facial expression recognition algorithm based on hierarchical deep neural network structure[J]. IEEE Access, 2019(7):41273-41285.
[16]	李校林, 钮海涛. 基于VGG-NET的特征融合面部表情识别[J]. 计算机工程与科学, 2020, 42(3):500-509.
	Li Xiaolin, Niu Haitao. Facial expression recognition using feature fusion based on VGG-NET[J]. Computer Engineering & Science, 2020, 42(3):500-509.
[17]	陈婷, 张亚南, 张赛, 等. 基于深度学习网络的小目标检测算法研究[J]. 电子设计工程, 2020, 28(23):84-87.
	Chen Ting, Zhang Yanan, Zhang Qian, et al. Research on small object detection algorithm based on deep learning network[J]. Electronic Design Engineering, 2020, 28(23):84-87.

输入	操作×n	卷积核	步长	通道数
224²×3	Con2d	3×3	2	32
112²×32	Invert_Res_Block×1	3×3	1	16
112²×16	Invert_Res_Block×2	3×3	2	24
56²×24	Invert_Res_Block×3	3×3	2	32
28²×32	Invert_Res_Block×4	3×3	2	64
28²×64	Invert_Res_Block×3	3×3	1	96
14²×96	Invert_Res_Block×3	3×3	2	160
7²×160	Invert_Res_Block×1	3×3	1	320
7²×320	Con2d	3×3	1	1 280
7²×1 280	Global_Avg_Pool	7×7	-	-
1²×1 280	Dense	100	-	-

Infant Expression Recognition Algorithm Based on MobileNetV2 and LBP Feature Fusion

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 17

References 17

Related Articles 15

Metrics

Comments

Recommended 10

[1]	HUANG Yuansheng,JIANG Yuqing,WANG Jing. Research on Adaptability Evaluation of Distribution Network Based on Improved TOPSIS-PSO-SVM [J]. Electronic Science and Technology, 2022, 35(6): 54-63.
[2]	SHEN Ningjing,YUAN Jian. Crowd Counting Algorithm Based on Residual Dense Connection and Attention Fusion [J]. Electronic Science and Technology, 2022, 35(6): 6-12.
[3]	Chaowei LIN,Feifei LI,Qiu CHEN. Globaland Local Scene Representation Method Based on Deep Convolutional Features [J]. Electronic Science and Technology, 2022, 35(4): 20-27.
[4]	Lingyu JI,Yongbin GAO,Chenglu ZHAO,Xianhua TANG,Kaicheng XU,Jiacheng XU. CTA Segmentation Algorithm of Abdominal Artery Based on 3D Fully Convolutional Network [J]. Electronic Science and Technology, 2022, 35(3): 38-44.
[5]	WU Weijia,YANG Jian,YUAN Tianchen,SHAO Zhihui. Research on Track Structure Damage Identification Based on Support Vector Machine [J]. Electronic Science and Technology, 2022, 35(2): 27-33.
[6]	LI Hui,WANG Yicheng. CNNCIFG-Attention Model for Text Sentiment Classifcation [J]. Electronic Science and Technology, 2022, 35(2): 46-51.
[7]	SHAO Zhihui,YANG Jian,YUAN Tianchen,WU Weijia. Sleeper Diseases Diagnosis Based on Permutation Entropy and Support Vector Machine [J]. Electronic Science and Technology, 2022, 35(2): 52-58.
[8]	ZHAO Chong,CHI Mengmeng,CHU Cong,ZHANG Peng. Research on Motion Simulation and Visual Recognition Algorithm of Guide Dog Walking Mechanism [J]. Electronic Science and Technology, 2021, 34(9): 66-72.
[9]	ZHAN Yinwei,ZHU Baiwan,YANG Zhuo. Research and Application of Vehicle Face Recognition Algorithm [J]. Electronic Science and Technology, 2021, 34(8): 1-7.
[10]	LIAO Luming,ZHANG Wei. Batch Mixed Training Traffic Sign Recognition Based on Improved VGG16 Network [J]. Electronic Science and Technology, 2021, 34(8): 8-13.
[11]	MA Lixin,DOU Chenfei,SONG Chencan,YANG Tianxiao. Insulator Nondestructive Testing Based on Feature Fusion CNN [J]. Electronic Science and Technology, 2021, 34(7): 26-30.
[12]	ZHANG Pengfei,WEI Cunwei,LIU Xiankang,LIU Anran,YANG Ou,LIN Yonglin. Research on Ship Target Recognition Method of Narrowband Radar [J]. Electronic Science and Technology, 2021, 34(6): 50-55.
[13]	GU Wei,LI Feifei,CHEN Qiu. Pedestrian Detection Algorithm Based on Multiple Feature Fusion [J]. Electronic Science and Technology, 2021, 34(5): 29-34.
[14]	LIU Shu,SHAO Jie,ZHANG Yiting,ZHANG Shanzhang. ECG Classification Based on Bispectrum and Spectral Features [J]. Electronic Science and Technology, 2021, 34(5): 42-46.
[15]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.