互惠双向生成对抗网络用于跨模态行人重识别

doi:10.19665/j.issn1001-2400.2021.02.026

西安电子科技大学学报 ›› 2021, Vol. 48 ›› Issue (2): 205-212.doi: 10.19665/j.issn1001-2400.2021.02.026

• 计算机科学与技术 • 上一篇

互惠双向生成对抗网络用于跨模态行人重识别

魏梓钰¹(),杨曦¹(),王楠楠¹(),杨东²(),高新波³()

1.西安电子科技大学通信工程学院,陕西西安 710071
2.西安空间无线电技术研究所,陕西西安 710100
3.重庆邮电大学图像认知重庆市重点实验室,重庆 400065

收稿日期:2020-06-12 修回日期:2020-01-06 出版日期:2021-04-20 发布日期:2021-04-28
通讯作者: 杨曦
作者简介:魏梓钰(1996—),女,西安电子科技大学博士研究生,E-mail: zywei_xd@stu.xidian.edu.cn|王楠楠(1986—),男,教授,E-mail: nnwang@xidian.edu.cn|杨东(1988—),男,高级工程师,E-mail: yangd504@126.com|高新波(1972—),男,教授,E-mail: gaoxb@cqupt.edu.cn,xbgao@mail.xidian.edu.cn
基金资助:
国家自然科学基金(61976166);国家自然科学基金(61772402);国家自然科学基金(61671339);国家自然科学基金(U1605252);国家自然科学基金(61772387);国家自然科学基金(61922066);国家自然科学基金(61876142);国家重点研发计划(2016QY01W0200);国家重点研发计划(2018AAA0103202);国家高层次人才专项支持计划(CS31117200001);陕西省创新能力支撑计划(2020KJXX-027);中央高校基本科研业务费(JB210115);陕西高校科协青年人才托举计划(20180104);陕西省重点研发计划(2021GY-030);西安电子科技大学研究生创新基金(5001-20109215456)

Reciprocal bi-directional generative adversarial network for cross-modal pedestrian re-identification

WEI Ziyu¹(),YANG Xi¹(),WANG Nannan¹(),YANG Dong²(),GAO Xinbo³()

1. School of Telecommunications Engineering,Xidian University,Xi’an 710071,China
2. Xi’an Institute of Space Radio Technology,Xi’an 710100,China
3. Chongqing Key Laboratory of Image Cognition,Chongqing University of Posts and Telecommunications,Chongqing 400065,China

Received:2020-06-12 Revised:2020-01-06 Online:2021-04-20 Published:2021-04-28
Contact: Xi YANG

摘要/Abstract

摘要：

为提高跨模态行人重识别的准确率,提出一种基于互惠双向生成对抗网络的跨模态行人重识别方法。首先,建立两个生成对抗网络以生成跨模态异质图像;其次,设计一种联合损失在可见光与红外图像相互转换过程中拉近隐藏空间特征的分布,促使网络生成更接近真实图像的伪异质图像;最后,通过将原始图像与生成的异质行人图像相结合并输入至区别性特征提取网络中,使得不同模态的图像统一至相同模态,消减了跨模态差异。利用表征学习与度量学习,使网络提取出更具有判别性的行人特征。通过在跨模态数据集SYSU-MM01和RegDB上做对比实验,分析了该方法在不同损失函数下的识别效果。对比于其他前沿跨模态行人重识别方法,这种方法具有更高准确率和更强鲁棒性。

关键词: 生成对抗网络, 图像转换, 特征提取, 跨模态行人重识别

Abstract:

To improve the accuracy of cross-modal pedestrian re-identification,a reciprocal bi-directional generative adversarial network-based method is proposed.First,we build two generative adversarial networks to generate cross-modal heterogeneous images.Second,an associated loss is designed to pull close the distribution of features in latent space during the image translation between visible and infrared images so as to help the networks generate fake heterogeneous images that have high similarity with the real images.Finally,by concatenating the original and generated heterogeneous pedestrian images into the discriminative feature extraction network,images from different modalities can be unified into a common modality,thus suppressing the cross-modal gap.Representation learning and metric learning are utilized to achieve more discriminative pedestrian features.Comparative experiments are conducted on SYSU-MM01 and RegDB datasets to analyze the accuracy with different loss functions.Compared with other state-of-the-art cross-modal pedestrian re-identification methods,the proposed method achieves a higher accuracy and stronger robustness.

Key words: generative adversarial networks, image translation, feature extraction, cross-modal pedestrianre-identification

中图分类号:

TN911.73

魏梓钰,杨曦,王楠楠,杨东,高新波. 互惠双向生成对抗网络用于跨模态行人重识别[J]. 西安电子科技大学学报, 2021, 48(2): 205-212.

WEI Ziyu,YANG Xi,WANG Nannan,YANG Dong,GAO Xinbo. Reciprocal bi-directional generative adversarial network for cross-modal pedestrian re-identification[J]. Journal of Xidian University, 2021, 48(2): 205-212.

图/表 7

图1

图2

图3

表1

图4

表2

图5

参考文献 15

[1]	罗浩, 姜伟, 范星, 等. 基于深度学习的行人重识别研究进展[J]. 自动化学报, 2019,45(11):2032-2049.
	LUO Hao, JIANG Wei, FAN Xing, et al. A Survey on Deep Learning Based Person Re-identification[J]. Acta Automatica Sinica, 2019,45(11):2032-2049.
[2]	朱莉, 赵俊, 傅应锴, 等. 一种红外热图像目标区域分割的深度学习算法[J]. 西安电子科技大学学报, 2019,46(4):107-114.
	ZHU Li, ZHAO Jun, FU Yingkai, et al. Deep Learning Algorithm for the Segmentation of the Interested Region of an Infrared Thermal Image[J]. Journal of Xidian University, 2019,46(4):107-114.
[3]	WU A, ZHENG W S, YU H X, et al. RGB-Infrared Cross-Modality Person Re-Identification[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017: 5390-5399.
[4]	NGUYEN D T, HONG H G, KIM K W, et al. Person Recognition System Based on a Combination of Body Images from Visible Light and Thermal Cameras[J]. Sensors, 2017,17(3):605.
[5]	YE M, WANG Z, LAN X, et al. Visible Thermal Person Re-Identification via Dual-Constrained Top-Ranking[C]//Proceedings of the 2018 International Joint Conferences on Artificial Intelligence. Stockholm:International Joint Conferences on Artificial Intelligence, 2018: 1092-1099.
[6]	HAO Y, WANG N, LI J, et al. HSME:Hypersphere Manifold Embedding for Visible Thermal Person Re-identification[C]//Proceedings of the 2019 33rd AAAI Conference on Artificial Intelligence. Palo Alto:AAAI Press, 2019: 8385-8392.
[7]	申海杰, 边倩, 陈晓范, 等. 生成对抗网络用于视频去模糊[J]. 西安电子科技大学学报, 2019,46(6):112-117.
	SHEN Haijie, BIAN Qian, CHEN Xiaofan, et al. Video Deblurring Using the Generative Adversarial Network[J]. Journal of Xidian University, 2019,46(6):112-117.
[8]	KNIAZ V V, KNYAZ V A, HLADUVKA J, et al. ThermalGAN:Multimodal Color-to-Thermal Image Translation for Person Re-Identification in Multispectral Dataset[C]//Lecture Notes in Computer Science:11134. Heidelberg:Springer Verlag, 2019: 606-624.
[9]	DAI P, JI R, WANG H, et al. Cross-Modality Person Re-Identification with Generative Adversarial Training[C]// Proceedings of the 2018 International Joint Conferences on Artificial Intelligence. Stockholm:International Joint Conferences on Artificial Intelligence, 2018: 677-683.
[10]	杜鹏, 宋永红, 张鑫瑶. 基于自注意力模态融合网络的跨模态行人再识别方法研究[J]. 自动化学报, 2019: 1-12.
	DU Peng, SONG Yonghong, ZHANG Xinyao. Self-Attention Cross-Modality Fusion Network for Cross-Modality Person Re-identification[J]. Acta Automatica Sinica, 2019: 1-12.
[11]	ZHU J Y, PARK T, ISOLA P, et al. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017: 2242-2251.
[12]	WANG Z, WANG Z, ZHENG Y, et al. Learning to Reduce Dual-level Discrepancy for Infrared-visible Person Re-identification[C]//Proceedings of the 2019 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2019: 618-626.
[13]	ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image Translation with Conditional Adversarial Networks[C]//Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017: 1125-1134.
[14]	LIAO S, HU Y, ZHU X, et al. Person Re-identification by Local Maximal Occurrence Representation and Metric Learning[C]//Proceedings of the 2015 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2015: 2197-2206.
[15]	DALAL N, TRIGGS B. Histograms of Oriented Gradients for Human Detection[C]// Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2005: 886-893.

Λ_rec	Λ_cyc	Λ_ass	Λ_id	Λ_metric	CMC-1	CMC-10	CMC-20	mAP
√	×	√	√	√	34.65	76.15	87.04	48.02
√	√	×	√	√	39.44	79.36	88.59	52.27
√	√	√	×	√	1.05	10.86	21.40	5.44
√	√	√	√	×	32.29	72.68	83.12	45.48
√	√	√	√	√	43.04	81.85	90.77	55.58

方法	SYSU-MM01				RegDB
方法	CMC-1	CMC-10	CMC-20	mAP	CMC-1	CMC-10	CMC-20	mAP
LOMO	1.75	14.14	26.63	3.48	0.85	2.47	4.10	2.28
HOG	2.76	18.25	31.91	4.24	13.49	33.22	43.66	10.31
One-stream	12.04	49.68	66.74	13.67	13.11	32.98	42.51	14.02
Two-stream	11.65	47.99	65.50	12.85	12.43	30.36	40.96	13.42
Zero-padding	14.80	54.12	71.33	15.59	17.75	34.21	44.35	18.90
BDTR	17.01	55.43	71.96	19.66	33.47	58.42	67.52	31.83
D-HSME	20.68	62.74	77.95	23.12	50.85	73.36	81.66	47.00
cmGAN	26.97	67.51	80.56	27.80
D²RL	28.90	70.60	82.40	29.20	43.40	66.10	76.30	44.10
自注意力模态融合	33.31	74.51	85.79	33.18
互惠双向网络(Ours)	43.04	81.85	90.77	55.58	73.79	89.81	93.25	68.96

互惠双向生成对抗网络用于跨模态行人重识别

Reciprocal bi-directional generative adversarial network for cross-modal pedestrian re-identification

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 15

相关文章 15

Metrics

本文评价

推荐阅读 10

[1]	于浩洋,尹良,李书芳,吕顺. 生成对抗网络小样本雷达调制信号识别算法[J]. 西安电子科技大学学报, 2021, 48(6): 96-104.
[2]	孙彦景,魏力,张年龙,云霄,董锴文,葛敏,程小舟,侯晓峰. 联合DD-GAN和全局特征的井下人员重识别方法[J]. 西安电子科技大学学报, 2021, 48(5): 201-211.
[3]	王军军,孙岳,李颖. 一种生成对抗网络的遥感图像去云方法[J]. 西安电子科技大学学报, 2021, 48(5): 23-29.
[4]	杨静波,赵启军,吕泽均. 维度情感模型下的表情图像生成及应用[J]. 西安电子科技大学学报, 2021, 48(5): 30-37.
[5]	刘洁怡,公茂果,詹涛,李豪,张明阳. 一种深度神经网络多站雷达系统干扰鉴别方法[J]. 西安电子科技大学学报, 2021, 48(2): 133-138.
[6]	郭泽坤,田隆,韩宁,王鹏辉,刘宏伟,陈渤. 采用CNN-SSD的雷达HRRP小样本目标识别方法[J]. 西安电子科技大学学报, 2021, 48(2): 7-14.
[7]	韩卓茜,王锋,陈沛,李卓伦. 一种历史特征辅助的模糊数据关联算法[J]. 西安电子科技大学学报, 2021, 48(2): 92-98.
[8]	王晓华,要鹏超,马丽萍,王文杰,张蕾. 车间环境下机器人语音控制的特征提取算法[J]. 西安电子科技大学学报, 2020, 47(2): 16-22.
[9]	张志远,刁英华. 结合社会特征和注意力的行人轨迹预测模型[J]. 西安电子科技大学学报, 2020, 47(1): 10-17.
[10]	杨晓莉,蔺素珍. 一种注意力机制的多波段图像特征级融合方法[J]. 西安电子科技大学学报, 2020, 47(1): 120-127.
[11]	申海杰,边倩,陈晓范,王振铎,田新志. 生成对抗网络用于视频去模糊[J]. 西安电子科技大学学报, 2019, 46(6): 112-117.
[12]	贾宇峰,马力. 条件约束下的自我注意生成对抗网络[J]. 西安电子科技大学学报, 2019, 46(6): 163-171.
[13]	王红艳,朱利民,张潘杰,李金屏. 一种利用外轮廓配准的活塞侧面缺陷检测方法[J]. 西安电子科技大学学报, 2019, 46(5): 75-83.
[14]	莫建文,徐凯亮,林乐平,欧阳宁. 结合互信息最大化的文本到图像生成方法[J]. 西安电子科技大学学报, 2019, 46(5): 180-188.
[15]	郭睿,臧博,彭树铭,邢孟道. 高分辨InSAR中的城市高层建筑特征提取[J]. 西安电子科技大学学报, 2019, 46(4): 137-143.