基于稠密轨迹特征的动态手势识别研究

doi:10.16180/j.cnki.issn1007-7820.2020.12.004

摘要/Abstract

摘要：

针对目前手势识别特征点检测的低效性,文中提出一种新的边缘特征点检测方法。文中方法通过借鉴稠密轨迹方法的特征描述方式,利用支持向量机分类学习的方法实现动态手势识别。该方法能够有效增加边缘轨迹的数量,从而给最终的识别带来裨益。利用剑桥大学手势数据集和谢菲尔德手势数据集进行性能评估,所提方法分别获得了99.11%与99.72%手势识别精度,体现了新算法在上述数据集中的优异性。

关键词: 手势识别, 边缘特征, 稠密光流, 边缘轨迹, 特征降维, Fisher向量, 支持向量机

Abstract:

In view of at the low performance of feature point detection for gesture recognition, the paper propose a new edge feature point detection method. By using the feature description method of the dense trajectories method, the method of support vector machine classification learning is uses to realize dynamic gesture recognition. This method increases the number of edge trajectories effectively, which brings benefits to the final recognition result. The method is evaluated in Cambridge University Gesture Dataset and Sheffield Gesture Dataset, and 99.11% and 99.72% gesture recognition accuracy is obtained respectively, which embodies the excellent performance of the algorithm in the above datasets.

Key words: gesture recognition, edge feature, dense optical flow, edge trajectories, feature reduction, Fisher vector, support vector machine

中图分类号:

TP391

王贺贺,李菲菲,陈虬. 基于稠密轨迹特征的动态手势识别研究[J]. 电子科技, 2020, 33(12): 17-21.

WANG Hehe,LI Feifei,CHEN Qiu. Research on Dynamic Gesture Recognition Based on Dense Trajectories Features[J]. Electronic Science and Technology, 2020, 33(12): 17-21.

图/表 7

图1

图2

图3

表1

图4

表2

表3

参考文献 33

[1]	Wang C, Liu Z, Chan S C. Superpixel-based hand gesture recognition with kinect depth camera[J]. IEEE Transactions on Multimedia, 2015,17(1):29-39. doi: 10.1109/TMM.2014.2374357
[2]	Ren Z, Yuan J, Meng J, et al. Robust part-based hand gesture recognition using kinect sensor[J]. IEEE Transactions on Multimedia, 2013,15(5):1110-1120. doi: 10.1109/TMM.2013.2246148
[3]	Hikawa H, Kaida K. Novel FPGA implementation of hand sign recognition system with SOM-hebb classifier[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2015,25(1):153-166. doi: 10.1109/TCSVT.2014.2335831
[4]	Marin G, Dominio F, Zanuttigh P. Hand gesture recognition with leap motion and kinect devices[C]. Paris:IEEE International Conference on Image Processing, 2014.
[5]	Kuznetsova A, Laura Leal-Taixé, Rosenhahn B. Real-time sign language recognition using a consumer depth camera[C]. Sydney:IEEE International Conference on Computer Vision Workshops, 2013.
[6]	Yao Y, Fu Y. Contour model-based hand-gesture recognition using the kinect sensor[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014,24(11):1935-1944. doi: 10.1109/TCSVT.2014.2302538
[7]	Prasuhn L, Oyamada Y, Mochizuki Y, et al. A HOG-based hand gesture recognition system on a mobile device[C]. Paris:IEEE International Conference on Image Processing, 2014.
[8]	Schramm R, Jung C R, Miranda E R. Dynamic time warping for music conducting gestures evaluation[J]. IEEE Transactions on Multimedia, 2015,17(2):243-255. doi: 10.1109/TMM.2014.2377553
[9]	Lian S, Hu W, Wang K. Automatic user state recognition for hand gesture based low-cost television control system[J]. IEEE Transactions on Consumer Electronics, 2014,60(1):107-115.
[10]	Sathayanarayana S, Satzoda R K, Carini A, et al. Towards automated understanding of student-tutor interactions using visual deictic gestures[C]. Columbus:IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2014.
[11]	程文山. 基于肤色分割和 Camshift 的手势识别研究[D]. 武汉:华中师范大学, 2009.
	Cheng Wenshan. The research of hand gesture recognition based on skin color and Camshift algorithm[D]. Wuhan: Central China Normal University, 2009.
[12]	Ruan X, Tian C. Dynamic gesture recognition based on improved DTW algorithm[C]. Beijing:IEEE International Conference on Mechatronics and Automation, 2015.
[13]	Zhang M, Wang B, Zhou S, et al. Dynamic gesture recognition based on edge feature enhancement using sobel operator[C]. Bournemouth:International Conference on Technologies for E-learning and Digital Entertainment, 2017.
[14]	Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004,60(2):91-110. doi: 10.1023/B:VISI.0000029664.99615.94
[15]	Wang H, Klaser A, Schmid C, et al. Action recognition by dense trajectories[C]. Providence:Computer Vision and Pattern Recognition, 2011.
[16]	Gunnar Farnebäck. Two-frame motion estimation based on polynomial expansion[C]. Halmstad:The Thirteeth Scandinavian Conference on Image Analysis, 2003.
[17]	Dollar P, Rabaud V, Cottrell G, et al. Behavior recognition via sparse spatio-temporal features[C]. Beijing:IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2005.
[18]	Dalal N, Triggs B. Histograms of oriented gradients for human detection[C]. San Diego:IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005.
[19]	Laptev I, Marszalek M, Schmid C, et al. Learning realistic human actions from movies[C]. Anchorage:IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2008.
[20]	Dalal N, Triggs B, Schmid C, et al. Human detection using oriented histograms of flow and appearance[C]. Graz: European Conference on Computer Vision, 2006.
[21]	Perronnin F, Sanchez J, Mensink T, et al. Improving the fisher kernel for large-scale image classification[C]. Heraklion:European Conference on Computer Vision, 2010.
[22]	Xu H, Tian Q, Wang Z, et al. A survey on aggregating methods for action recognition with dense trajectories[J]. Multimedia Tools and Applications, 2016,75(10):5701-5717. doi: 10.1007/s11042-015-2536-2
[23]	Fan R, Chang K, Hsieh C, et al. Liblinear: a library for large linear classification[J]. Journal of Machine Learning Research, 2008,9(8):1871-1874.
[24]	Kim T, Wong S, Cipolla R, et al. Tensor canonical correlation analysis for action classification[C]. Minneapolis:IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2007.
[25]	Liu L, Shao L. Learning discriminative representations from RGB-D video data[C]. Beijing:International Joint Conference on Artificial Intelligence, 2013.
[26]	Shao L, Cai Z, Liu L, et al. Performance evaluation of deep feature learning for RGB-D image/video classification[J]. Information Sciences, 2017,385(C):266-283.
[27]	Liu M, Liu H, Chen C. 3D action recognition using multi-scale energy-based global ternary image[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018,28(8):1824-1838. doi: 10.1109/TCSVT.2017.2655521
[28]	Reza A, Maryam A A, Shohreh K, et al. Dynamic 3D hand gesture recognition by learning weighted depth motion maps[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019,29(6):1729-1740. doi: 10.1109/TCSVT.76
[29]	Xing M, Hu J, Feng Z, et al. Dynamic hand gesture recognition using motion pattern and shape descriptors[J]. Multimedia Tools and Applications, 2019,78(8):10649-10672. doi: 10.1007/s11042-018-6553-9
[30]	Hsieh C, Lin W. Video-based human action and hand gesture recognition by fusing factored matrices of dual tensors[J]. Multimedia Tools and Applications, 2017,76(6):7575-7594. doi: 10.1007/s11042-016-3407-1
[31]	Patil A R, Subbaraman S. A spatiotemporal approach for vision-based hand gesture recognition using Hough transform and neural network[J]. Signal, Image and Video Processing, 2019,13(2):413-421. doi: 10.1007/s11760-018-1370-1
[32]	Li J, Huai H, Gao J, et al. Spatial-temporal dynamic hand gesture recognition via hybrid deep learning model[J]. Journal on Multimodal User Interfaces, 2019(9):1-9.
[33]	Tang H, Liu H, Xiao W, et al. Fast and robust dynamic hand gesture recognition via key frames extraction and feature fusion[J]. Neurocomputing, 2019,33(1):424-433.

数据集	类别/类	视频数/个	训练集/个	测试集/个
剑桥大学	9	900	450	225
谢菲尔德	10	1 080	720	360

方法	精度/%
文献[26]	93.30
文献[27]	93.88
文献[28]	97.31
文献[29]	98.70
本文	99.72

方法	精度/%
文献[30]	93.61
文献[31]	96.00
文献[32]	97.78
文献[33]	98.23
本文	99.11