结合社会特征和注意力的行人轨迹预测模型

doi:10.19665/j.issn1001-2400.2020.01.002

摘要/Abstract

摘要：

针对社会生成对抗网络行人交互特征简单且无法充分利用行人交互信息的问题,提出一种结合社会特征和注意力机制的行人轨迹预测模型。采用生成对抗网络模型,其中生成器使用编码器-解码器结构,中间加入注意力模块,并且设置3种社会特征以丰富行人交互信息。辅助注意力模块对同一场景中的行人进行影响力分配,使网络可以充分利用行人交互信息,提升模型的准确性。多个数据集上的实验结果表明,该模型较之前基于池化模块行人轨迹预测模型的准确率平均提高15%,且在行人密集、非直线轨迹多的场景中准确率提升34%,效果更加明显。

关键词: 轨迹生成, 生成对抗网络, 注意力机制, 长短时记忆网络, 行人交互

Abstract:

To address the problems that the pedestrian interaction feature of the Social GAN is simple and that it cannot make full use of the most of pedestrian interaction information, this paper proposes a pedestrian trajectory prediction model with social features and attention mechanism. This model adapts the structure of generative adversarial networks. The generator adapts an encoder-decoder model and the attention model is put between encoder and decoder. Three social features are set to enrich pedestrian interaction information which assists the attention module to make full use of the most of pedestrian interaction information by allocating the influence of pedestrians in the scene, so that the accuracy of the model is improved. Experimental results on multiple datasets show that the accuracy of this model in the pedestrian trajectory prediction task is increased by 15% compared with the previous pedestrian trajectory prediction model based on the pooling module. The improvement effect is most obvious in scenes with dense pedestrians and lots of non-straight tracks, with the accuracy increased by 34%.

Key words: trajectory generation, generative adversarial networks, attention mechanism, long short-term memory, pedestrian interaction

中图分类号:

TP391

张志远,刁英华. 结合社会特征和注意力的行人轨迹预测模型[J]. 西安电子科技大学学报, 2020, 47(1): 10-17.

ZHANG Zhiyuan,DIAO Yinghua. Pedestrian trajectory prediction model with social features and attention[J]. Journal of Xidian University, 2020, 47(1): 10-17.

图/表 7

图1

图2

图3

表1

表2

表3

图4

参考文献 17

[1]	LUO Y, CAI P, BERA A , et al. PORCA: Modeling and Planning for Autonomous Driving among Many Pedestrians[J]. IEEE Robotics and Automation Letters, 2018,3(4):3418-3425.
[2]	CHENG J, CHENG H, MENG M Q H , et al. Autonomous Navigation by Mobile Robots in Human Environments: a Survey [C]//Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetics. Piscataway: IEEE, 2018: 1981-1986.
[3]	CHOI W, SAVARESE S . A Unified Framework for Multi-target Tracking and Collective Activity Recognition [C]//Lecture Notes in Computer Science: 7575. Heidelberg: Springer Verlag, 2012: 215-230.
[4]	LIPTON Z C, BERKOWITZ J, ELKAN C . A Critical Review of Recurrent Neural Networks for Sequence Learning[J/OL].[2019-7-16]. Computer Science, 2015.
[5]	CHUNG J, KASTNER K, DINH L , et al. A Recurrent Latent Variable Model for Sequential Data [C]//Advances in Neural Information Processing Systems 28: 2015. Vancouver: Neural Information Processing Systems Foundation, 2015: 2980-2988.
[6]	LIU J, SHAHROUDY A, XU D , et al. Spatio-temporal LSTM with Trust Gates for 3d Human Action Recognition [C]//Lecture Notes in Computer Science: 9907. Heidelberg: Springer Verlag, 2016: 816-833.
[7]	ALAHI A, GOEL K, RAMANATHAN V , et al. Social LSTM: Human Trajectory Prediction in Crowded Spaces[C]//Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 961-971.
[8]	VEMULA A, MUELLING K, OH J . Social Attention: Modeling Attention in Human Crowds[C]//Proceedings of the 2018 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2018: 4601-4607.
[9]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M , et al. Generative Adversarial Nets[C]//Advances in Neural Information Processing Systems 27: 3. Vancouver: Neural Information Processing Systems Foundation, 2014: 2672-2680.
[10]	陈晓范, 申海杰, 边倩 , 等. 结合注意力机制的人脸超分辨率重建[J]. 西安电子科技大学学报, 2019,46(3):148-153.
	CHEN Xiaofan, SHEN Haijie, BIAN Qian , et al. Face Image Super-resolution with an Attention Mechanism[J]. Journal of Xidian University, 2019,46(3):148-153.
[11]	YU L, ZHANG W, WANG J , et al. Seqgan:Sequence Generative Adversarial Nets with Policy Gradient[C]//Proceedings of the 2017 31st AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2017: 2852-2858.
[12]	GUPTA A, JOHNSON J, FEI-FEI L , et al. Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks[C]//Proceedings of the 2018 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 2255-2264.
[13]	VASWANI A, SHAZEER N, PARMAR N , et al. Attention is All You Need[C]//Advances in Neural Information Processing Systems 30: 2017. Vancouver: Neural Information Processing Systems Foundation, 2017: 5998-6008.
[14]	LUONG M T, PHAM H, MANNING C D . Effective Approaches to Attention-based Neural Machine Translation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. New York:Association for Computational Linguistics, 2015: 1412-1421.
[15]	PELLEGRINI S, ESS A, VAN GOOL L . Improving Data Association by Joint Modeling of Pedestrian Trajectories and Groupings[C]//Lecture Notes in Computer Science: 6311. Heidelberg: Springer Verlag, 2010: 452-465.
[16]	LERNER A, CHRYSANTHOU Y, LISCHINSKI D . Crowds by Example[J]. Computer Graphics Forum, 2007,26(3):655-664.
[17]	LEE N, CHOI W, VERNAZA P , et al. Desire: Distant Future Prediction in Dynamic Scenes with Interacting Agents[C]//Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2165-2174.

数据集	ETH	HOTEL	UNIV	ZARA1	ZARA2
数据量	5492	6543	21813/17953	5153	9722
截取帧数	876	1168	444/541	872	1052
行人总数	360	389	415/434	148	204
一帧最大行人数	27	18	75/52	20	20
一帧最小行人数	1	1	21/13	1	1
每帧平均行人数	6.27	5.60	49.13/33.18	5.91	9.24
单个行人存在最大帧数	114	100	352/289	197	584
单个行人存在最小帧数	2	2	4/5	10	7
行人存在平均帧数	15.26	16.82	52.56/41.37	34.82	47.66

评价指标	数据集	Linear	LSTM	S-LSTM	SGAN	+社会特征	+注意力模块	文中方法
ADE	ETH	0.84	0.70	0.73	0.69	0.67	0.65	0.63
	HOTEL	0.35	0.55	0.49	0.49	0.46	0.35	0.34
	UNIV	0.56	0.36	0.41	0.40	0.38	0.36	0.33
	ZARA1	0.41	0.25	0.27	0.22	0.21	0.21	0.20
	ZARA2	0.53	0.31	0.33	0.22	0.22	0.21	0.20
平均ADE		0.54	0.43	0.45	0.42	0.39	0.36	0.34

评价指标	数据集	Linear	LSTM	S-LSTM	SGAN	+社会特征	+注意力模块	文中方法
FDE	ETH	1.60	1.45	1.48	1.45	1.33	1.38	1.24
	HOTEL	0.60	1.17	1.01	1.01	0.86	0.70	0.62
	UNIV	1.01	0.77	0.84	0.82	0.78	0.69	0.69
	ZARA1	0.74	0.53	0.56	0.43	0.41	0.41	0.40
	ZARA2	0.95	0.65	0.70	0.44	0.43	0.42	0.42
平均FDE		0.98	0.91	0.91	0.83	0.76	0.72	0.67