Electronic Science and Technology ›› 2021, Vol. 34 ›› Issue (8): 14-18.doi: 10.16180/j.cnki.issn1007-7820.2021.08.003

Previous Articles     Next Articles

Workflow Recognition Based on Temporal Action Detection

WANG Qingwen,HU Haiyang   

  1. Computer & Software School, Hangzhou Dianzi University,Hangzhou 310018,China
  • Received:2020-03-26 Online:2021-08-15 Published:2021-08-17
  • Supported by:
    National Natural Science Foundation of China(61572162);National Natural Science Foundation of China(61272188);National Natural Science Foundation of China(61702144);Zhejiang Provincial Key Science and Technology Project Foundation(2018C01012);Natural Science Foundation of Zhejiang Province(LQ17F020003)


In a complex manufacturing environment, it is difficult for workflow recognition based on action recognition to localize the start and end times of each activity in a workflow from the video. In order to localize the activities temporally in the workflow from the video, according to R-C3D, a workflow recognition framework based on temporal action detection is proposed. The proposed workflow recognition method introduces a random sparse sampling strategy to reduce the redundancy between adjacent frames, and uses Res3D network to extract spatio-temporal features in the video. In addition, Soft-NMS strategy is used to eliminate the highly overlapping and low confidence proposals. Experiment results show that compared with R-C3D, the proposed method can improve the speed of training and detecting about 80% and 85%, respectively without loss of accuracy, proving the effectiveness of the method for workflow identification.

Key words: intelligent manufacturing, workflow recognition, action recognition, temporal action detection, temporal localization, sparse sampling, spatio-temporal features, Soft-NMS

CLC Number: 

  • TP312