一种识别表情序列的卷积神经网络

doi:10.3969/j.issn.1001-2400.2018.01.027

Abstract

Abstract:

As an important part of the human-computer interaction system, facial expression recognition has been a hot research field. The convolutional neural network cannot recognize expression sequence although it can train the classification features automatically for the reason that the direction of feature training need to be specified manually. In order to solve this problem, this paper improves the network structure, and proposes a multi convolutional network fusion method that can be used to identify the expression sequences containing multiple frames. First, we construct a number of convolutional neural networks, so that each network processes one frame, and then merge the results in the merge layer, and finally pass the softmax classifier to give the identity result. On the CK+facial expression database, experiments are carried out on the 3rd, 4th and 5th frames of expression sequences, and a high recognition rate is obtained for all experiments.

Key words: facial expression recognition, convolutional neural network, deep learning, multi network convergence

ZHANG Jingang;FANG Yuan;YUAN Hao;WANG Shuzhen. Multiple convolutional neural networks for facial expression sequence recognition[J].Journal of Xidian University, 2018, 45(1): 150-155.

References

［1］ GHIMIRE D, LEE J. Geometric Feature-based Facial Expression Recognition in Image Sequences Using Multi-class AdaBoost and Support Vector Machines［J］. Sensors, 2013, 13(6): 7714-7734.
［2］ PANNING A, AL-HAMADI A K, NIESE R, et al. Facial Expression Recognition Based on Haar-like Feature Detection［J］. Pattern Recognition and Image Analysis, 2008, 18(3): 447-452.
［3］ GU W, XIANG C, VENKATESH Y V, et al. Facial Expression Recognition Using Radial Encoding of Local Gabor Features and Classifier Synthesis［J］. Pattern Recognition, 2012, 45(1): 80-91.
［4］ SHAN C, GONG S, MCOWAN P W. Facial Expression Recognition Based on Local Binary Patterns: a Comprehensive Study［J］. Image and Vision Computing, 2009, 27(6): 803-816.
［5］ MARTIN C, WERNER U, GROSS H M. A Real-time Facial Expression Recognition System Based on Active Appearance Models Using Gray Images and Edge Images［C］//Proceedings of the 2008 8th IEEE International Conference on Automatic Face and Gesture Recognition. Piscataway: IEEE, 2008: 4813412.
［6］ LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based Learning Applied to Document Recognition［J］. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
［7］刘如意, 宋建锋, 权义宁, 等. 一种自动的高分辨率遥感影像道路提取方法［J］. 西安电子科技大学学报, 2017, 44(1): 100-105.
LIU Ruyi, SONG Jianfeng, QUAN Yining, et al. Automatic Road Extraction Method for High-resolution Remote Sensing Images［J］. Journal of Xidian University, 2017, 44(1): 100-105.
［8］史鹤欢, 许悦雷, 马时平, 等. PCA预训练的卷积神经网络目标识别算法［J］. 西安电子科技大学学报, 2016, 43(3): 161-166.
SHI Hehuan, XU Yuelei, MA Shiping, et al. Convolutional Neural Networks Recognition Algorithm Based on PCA［J］. Journal of Xidian University, 2016, 43(3): 161-166.
［9］ KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet Classification with Deep Convolutional Neural Networks［C］//Proceedings of the Advances in Neural Information Processing Systems. Vancouver: Neural Information Processing System Foundation, 2012:1097-1105.
［10］ SZEGEDY C, LIU W, JIA Y, et al. Going Deeper with Convolutions［C］//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 1-9.
［11］ HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition［C］//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
［12］ LUCEY P, COHN J F, KANADE T, et al. The Extended Cohn-Kanade Dataset (CK+): a Complete Dataset for Action Unit and Emotion-specified Expression［C］//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2010: 94-101.

[1]	LIU Jiawei,ZHANG Wenhui,KOU Xiaoli,LI Yanni. Harnessing adversarial examples via input denoising and hidden information restoring [J]. Journal of Xidian University, 2021, 48(6): 23-31.
[2]	YU Haoyang,YIN Liang,LI Shufang,LV Shun. Recognition algorithm for the little sample radar modulation signal based on the generative adversarial network [J]. Journal of Xidian University, 2021, 48(6): 96-104.
[3]	LI Peng,FENG Cunqian,XU Xuguang,TANG Zixiang. Ballistic target fretting classification network based on Bayesian optimization [J]. Journal of Xidian University, 2021, 48(5): 139-148.
[4]	SUN Yanjing,WEI Li,ZHANG Nianlong,YUN Xiao,DONG Kaiwen,GE Min,CHENG Xiaozhou,HOU Xiaofeng. Person re-identification method combining the DD-GAN and Global feature in a coal mine [J]. Journal of Xidian University, 2021, 48(5): 201-211.
[5]	YAN Jia,CAO Yudong,REN Jiaxing,CHEN Donghao,LI Xiaohui. Deep asymmetric compression Hashing algorithm [J]. Journal of Xidian University, 2021, 48(5): 212-221.
[6]	YANG Jingbo,ZHAO Qijun,LYU Zejun. Synthesis of the expression image and its application under the dimentional emotion model [J]. Journal of Xidian University, 2021, 48(5): 30-37.
[7]	NING Yang,DU Jianchao,HAN Shuo,YANG Chuankai. Fire segmentation based on the improved DeeplabV3+ and the analytical method for fire development [J]. Journal of Xidian University, 2021, 48(5): 38-46.
[8]	ZHOU Peng,YANG Jun. Semantic segmentation of remote sensing images based on neural architecture search [J]. Journal of Xidian University, 2021, 48(5): 47-57.
[9]	QI Yanjun,KONG Yueping,WANG Jiajing,ZHU Xudong. Gait recognition method combining LSTM and CNN [J]. Journal of Xidian University, 2021, 48(5): 78-85.
[10]	YANG Yunhang,MIN Lianquan. Multi-scalefusion sketch recognition model by dilated convolution [J]. Journal of Xidian University, 2021, 48(5): 92-99.
[11]	CHEN Changchuan,WANG Haining,HUANG Lian,HUANG Tao,LI Lianjie,HUANG Xiangkang,DAI Shaosheng. Facial expression recognition based on local representation [J]. Journal of Xidian University, 2021, 48(5): 100-109.
[12]	SONG Jianfeng,MIAO Qiguang,WANG Chongxiao,XU Hao,YANG Jin. Multi-scale single object tracking based on the attention mechanism [J]. Journal of Xidian University, 2021, 48(5): 110-116.
[13]	ZHANG Yuhao,CHENG Peitao,ZHANG Shuhao,WANG Xiumei. Lightweight image super-resolution with the adaptive weight learning network [J]. Journal of Xidian University, 2021, 48(5): 15-22.
[14]	HUI Haisheng,ZHANG Xueying,WU Zelin,LI Fenglian. Method for stroke lesion segmentation using the primary-auxiliary path attention compensation network [J]. Journal of Xidian University, 2021, 48(4): 200-208.
[15]	SUN Haojie,LI Miaoyu,ZHANG Panpan,XU Pengfei. Self-supervised facial asymmetry learning for automatic evaluation of facial paralysis [J]. Journal of Xidian University, 2021, 48(3): 115-122.

Multiple convolutional neural networks for facial expression sequence recognition

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 10