Journal of Xidian University

Previous Articles     Next Articles

Real time feature extraction method for complex industrial big data

KONG Xianguang;ZHANG Xiong;MA Hongbo;CHANG Jiantao;NIU Meng   

  1. (Industrial Big Data Technology Research Center, Xidian Univ., Xi'an  710071, China)
  • Received:2015-08-13 Online:2016-10-20 Published:2016-12-02
  • Contact: KONG Xianguang


Industrial big data have the traits of big volume, multi-sources, continuous sampling and low value density, which results in high complexity, real-time and high abnormality. Traditional feature extraction methods cannot meet the real-time requirements of complex industrial big data. In addition, the processing method for industrial big data is different from the internet data stream processing method, which has a higher accuracy requirement. Therefore, this paper proposes a robust incremental on-line feature extraction method as the Robust Incremental Principal Component Analysis. It uses the sliding window to update new coming data dynamically and filter the abnormal data in windows, then the incremental principal component analysis is implemented on data in windows in order to meet the accuracy and real-time requirements of industrial big data processing. Experimental results show that the proposed method can effectively extract the data stream in real time with high accuracy.

Key words: industrial big data, real-time and robustness, sliding window, principal component analysis, outlier detection, feature extraction