双向长短期记忆网络的时间序列预测方法

doi:10.19665/j.issn1001-2400.20231205

Abstract

Abstract:

Time series prediction means the use of historical time series to predict a period of time in the future,so as to formulate corresponding strategies in advance.At present,the categories of time series are complex and diverse.However,existing time series prediction models cannot achieve stable prediction results when faced with multiple types of time series data.The application requirements of complex time series data prediction in reality are difficult to simultaneously meet.To address the problem,a time series prediction method is proposed based on the Bidirectional Long and Short-term Memory(BLSTM) with the attention mechanism.The improved forward and backward propagation mechanisms are used to extract temporal information.The future temporal information is inferred through an adaptive weight allocation strategy.Specifically,an improved BLSTM is proposed to extract deep time series features and explore temporal dependencies of context by combining BLSTM and Long Short-term Memory(LSTM) networks,on the basis of which the proposed temporal attention mechanism is fused to achieve adaptive weighting of deep time series features,which improves the saliency expression ability of deep time series features.Experimental results demonstrate that the proposed method has a superior prediction performance in comparison with some representative methods in multiple time series datasets of different categories.

Key words: time series, Bidirectional Long Short-Term Memory, Long Short-Term Memory, attention mechanism, deep learning

CLC Number:

TP391.41

GUAN Yepeng, SU Guangyao, SHENG Yi. Time series prediction method based on the bidirectional long short-term memory network[J].Journal of Xidian University, 2024, 51(3): 103-112.

Figures/Tables 12

References 22

[1]	KOWSARI K, MEIMANDI K J, HEIDARYSAFA M, et al. Text Classification Algorithms:A Survey[J]. Information, 2019, 10(4):150-172.
[2]	DURAIRAJ D M, MOHAN B H K. A Convolutional Neural Network Based Approach to Financial Time Series Prediction[J]. Neural Computing and Applications, 2022, 34(16):13319-13337.
[3]	CAMASTRA F, CAPONE V, CIARAMELLA A, et al. Predictionof Environmental Missing Data Time Series by Support Vector Machine Regression and Correlation Dimension Estimation[J]. Environmental Modelling & Software, 2022, 150:1043-1053.
[4]	刘惠, 董锡耀, 杨志涵. 融合Stacking框架的BiGRU-LGB云负载预测模型[J]. 西安电子科技大学学报, 2023, 50(3):83-94.
	LIU Hui, DONG Xiyao, YANG Zhihan. Bigru-LGB Cloud Load Prediction Model Incorporating Stacking Framework[J]. Journal of Xidian University, 2023, 50(3):83-94.
[5]	张梦迪, 徐庆, 刘振鸿, 等. 基于动态滑动窗口BP神经网络的水质时间序列预测[J]. 环境工程技术学报, 2022, 12(3):809-815.
	ZHANG Mengdi, XU Qing, LIU Zhenhong, et al. Water Quality Time Series Prediction Based on Dynamic Sliding Window BP Neural Network[J]. Journal of Environmental Engineering Technology, 2022, 12(3):809-815.
[6]	MOHANTY M K, THAKURTA P K G, KAR S. Agricultural Commodity Price Prediction Model:A Machine Learning Framework[J]. Neural Computing and Applications, 2023, 35(20):15109-15128.
[7]	YULE G U. On A Method of Investigating Periodicities Disturbed Series,with Special Reference to Wolfer's Sunspot Numbers[J]. Philosophical Transactions of the Royal Society of London.Series A,Containing Papers of a Mathematical or Physical Character, 1927, 226(636-646):267-298.
[8]	WALKER G T. On Periodicity in Series of Related Terms[J]. Proceedings of the Royal Society of London.Series A,Containing Papers of a Mathematical and Physical Character, 1931, 131(818):518-532.
[9]	CHO C, KWON K, WU C. On Weather Data-Based Prediction of Gamma Exposure Rates Using Gradient Boosting Learning for Environmental Radiation Monitoring[J]. Sensors, 2022, 22(18):7062.
[10]	LIU S, FU B, WANG W, et al. Dynamic Sepsis Prediction for Intensive Care Unit Patients Using Xgboost-Based Model with Novel Time-Dependent Features[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 26(8):4258-4269.
[11]	SIŁKA J, WIECZOREK M, WOŁNIAK M. Recurrent Neural Network Model for High-Speed Train Vibration Prediction from Time Series[J]. Neural Computing and Applications,2022,34(16):13305-13318.
[12]	HOCHREITER S, SCHMIDHUBER J. Long Short-Term Memory[J]. Neural Computation, 1997, 9(8):1735-1780. doi: 10.1162/neco.1997.9.8.1735 pmid: 9377276
[13]	HAFEZI L, REZAEIAN M. Neural Architecture for Persian Named Entity Recognition[C]//Proceedings of 4th Iranian Conference on Signal Processing and Intelligent Systems. Piscataway:IEEE, 2018:61-64.
[14]	KAUSHIK P, GUPTA A, ROY P P, et al. EEG-Based Age and Gender Prediction Using Deep BLSTM-LSTM Network Model[J]. IEEE Sensors Journal, 2019, 19(7):2634-2641. doi: 10.1109/JSEN.2018.2885582
[15]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All You Need[J]. Advances in Neural Information Processing Systems, 2017, 30:2171-2184.
[16]	KITAEV N, KAISER Ł, LEVSKAYA A. Reformer:The Efficient Transformer[C]//Proceedings of 9th International Conference on Learning Representations. La Jolla: ICLR, 2020:1-12.
[17]	ZERVEAS G, JAYARAMAN S, PATEL D, et al. A Transformer-Based Framework for Multivariate Time Series Representation Learning[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. New York: ACM, 2021:2114-2124.
[18]	ZHOU H, ZHANG S, PENG J, et al. Informer:Beyond Efficient Transformer for Long Sequence Time-Series Forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2021:11106-11115.
[19]	LIU G, GUO J. Bidirectional LSTM with Attention Mechanism and Convolutional Layer for Text Classification[J]. Neurocomputing, 2019, 337:325-338.
[20]	PRADHAN T, KUMAR P, PAL S. CLAVER:An Integrated Framework of Convolutional Layer,Bidirectional LSTM with Attention Mechanism based Scholarly Venue Recommendation[J]. Information Sciences, 2021, 559:212-235.
[21]	AGARWAL N, BRUKHIM N, HAZAN E, et al. Boosting for Control of Dynamical Systems[C]//Proceedings of International Conference on Machine Learning. New York: PMLR, 2020:96-103.
[22]	GASPARIN A, LUKOVIC S, ALIPPI C. Deep Learning for Time Series Forecasting:The Electric Load Case[J]. CAAI Transactions on Intelligence Technology, 2022, 7(1):1-25.

数据集名称	类型	特征数	数据集长度
Badminton	国际羽毛球比赛数据	4	3 155 537
AirQuality^[21]	意大利空气质量数据	12	9 538
Power^[22]	法国家庭用电量数据	8	2 075 260

特征	值	取值数量
pos	中中场,中前场,中后场…	9
hand	反手,头顶,正手	3
tec	劈吊,勾,吊,抽,接杀挑…	19
path	中路,斜线,直线	3

网络模型	评价指标
网络模型	MAE↓	MSE↓	RMSE↓	PCC↑
LightGBM^[9]	0.438	0.558	0.491	0.704
LSTM^[12]	0.243	0.372	0.342	0.883
BLSTM^[13]	0.227	0.327	0.296	0.896
BLSTM-L^[14]	0.226	0.324	0.290	0.905
Transformer^[17]	0.232	0.387	0.305	0.887
Att-BLSTM^[19]	0.228	0.326	0.297	0.899
TA-LSTM(文中方法)	0.223	0.323	0.281	0.904

网络模型	AirQuality^[21]				Power^[21]
网络模型	MAE↓	MSE↓	RMSE↓	PCC↑	MAE↓	MSE↓	RMSE↓	PCC↑
LightGBM^[9]	0.649	0.727	0.693	0.803	0.450	0.534	0.478	0.716
LSTM^[12]	0.423	0.670	0.646	0.910	0.163	0.391	0.319	0.823
BLSTM^[13]	0.426	0.678	0.651	0.902	0.179	0.424	0.345	0.826
BLSTM-L^[14]	0.432	0.674	0.664	0.878	0.177	0.407	0.332	0.816
Transformer^[17]	0.461	0.809	0.708	0.901	0.184	0.390	0.350	0.824
Att-BLSTM^[19]	0.488	0.867	0.766	0.837	0.174	0.381	0.320	0.827
TA-LSTM(文中方法)	0.397	0.596	0.626	0.922	0.166	0.370	0.318	0.832

方法	Badminton
方法	MSE↓	MAE↓	RMSE↓	PCC↑
(w/o) Time Attention	0.236	0.365	0.301	0.890
(w/o) BLSTM	0.228	0.325	0.293	0.901
(w/o) LSTM	0.225	0.325	0.297	0.904
TA-LSTM	0.223	0.323	0.281	0.904

Time series prediction method based on the bidirectional long short-term memory network

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 22

Related Articles 15

Metrics

Comments

Recommended 0

[1]	XIA Yilan, WANG Xiumei, CHENG Peitao. Texture-aware video inpainting algorithm based on the multi-attention mechanism [J]. Journal of Xidian University, 2024, 51(3): 136-146.
[2]	WANG Jing, HE Miaomiao, DING Jianli, LI Yonghua. Spatial-temporal graph convolutional networks foranomaly detection in multivariate time series [J]. Journal of Xidian University, 2024, 51(3): 170-181.
[3]	HENG Hongjun, YU Longwei. Time series anomaly detection based on multi-scale feature information fusion [J]. Journal of Xidian University, 2024, 51(3): 203-214.
[4]	HE Wangpeng, HU Deshun, LI Cheng, ZHOU Yue, GUO Baolong. Siamese network tracking using template updating and trajectory prediction [J]. Journal of Xidian University, 2024, 51(3): 46-54.
[5]	GE Yunfeng, LI Hongyan, SHI Keyi. A self-attention sequential model for long-term prediction of video streams [J]. Journal of Xidian University, 2024, 51(3): 88-102.
[6]	LIU Wei, WANG Mengyang, BAI Baoming. Efficient semantic communication method for bandwidth constrained scenarios [J]. Journal of Xidian University, 2024, 51(3): 9-18.
[7]	LIU Zhenyan, ZHANG Hua, LIU Yong, YANG Libo, WANG Mengdi. Efficient seed generation method for software fuzzing [J]. Journal of Xidian University, 2024, 51(2): 126-136.
[8]	ZHAI Fengwen, SUN Fanglin, JIN Jing. Study of EEG classification of depression by multi-scale convolution combined with the Transformer [J]. Journal of Xidian University, 2024, 51(2): 182-195.
[9]	DING Xinmiao, WANG Jiaxing, GUO Wen. Three-dimensional attention-enhanced algorithm for violence scene detection [J]. Journal of Xidian University, 2024, 51(1): 114-124.
[10]	LIU Bochong, CAI Huaiyu, WANG Yi, CHEN Xiaodong. Self-supervised contrastive representation learning for semantic segmentation [J]. Journal of Xidian University, 2024, 51(1): 125-134.
[11]	ZHONG Hao, BIAN Shan, WANG Chuntao. Real world image tampering localization combining the self-attention mechanism and convolutional neural networks [J]. Journal of Xidian University, 2024, 51(1): 135-146.
[12]	ZHENG Kengtao, LI Bin, ZENG Jinhua. Document image forgery localization and desensitization localization using the attention mechanism [J]. Journal of Xidian University, 2023, 50(6): 207-218.
[13]	XIONG Jingwei, PAN Jifei, BI Daping, DU Mingyang. Multi-scale convolutional attention network for radar behavior recognition [J]. Journal of Xidian University, 2023, 50(6): 62-74.
[14]	ZHU Guangming,LU Zijie,FENG Jiawei,ZHANG Xiangdong,ZHANG Fengjun,NIU Zuoyuan,ZHANG Liang. Cause-effectgraph enhanced APT attack detection algorithm [J]. Journal of Xidian University, 2023, 50(5): 107-117.
[15]	HOU Yue,ZHENG Xin,HAN Chengyan. Traffic flow prediction method for integrating longitudinal and horizontal spatiotemporal characteristics [J]. Journal of Xidian University, 2023, 50(5): 65-74.