利用混合双通路神经网络的跨模态行人重识别

doi:10.19665/j.issn1001-2400.2021.05.022

Abstract

Abstract:

An infrared image can effectively make up for the shortcomings of single-modality visible-light image data under low illumination conditions.Therefore,the study of cross-modality Visible-to-Infrared person re-identification will provide a strong technical support for constructing an intelligence video surveillance system under various lighting conditions.The key for cross-modality person re-identification is to construct a unified shared feature representation among multi-modal data,which needs to effectively distinguish the modal-shared/modal-specific feature information in the cross-modal data.Based on this,this paper proposes a cross-modality person re-identification method based on a hybrid dual-channel neural network.This method deeply analyzes the influence of the parameter-shared layer and the non-shared parameter layer on the cross-modality person re-identification model in the hybrid dual-channel neural network architecture.Besides cross-entropy loss,we also use the intra-class distribution and inter-class correlation constraints in the loss function to further improve the re-identification performance.In the optimization process,we effectively utilize the adaptive learning rate adjustment strategy to improve the feature learning capability of the neural network architecture.Experimental results illustrate the effectiveness of the proposed method.Also,we obtain a superior performance on the two widely used cross-modality person re-identification benchmark datasets,SYSU-MM01 and RegDB.

Key words: person re-identification, cross-modality, two-stream neural network, shared feature representation space

CLC Number:

TP391.4

CHENG De,HAO Yi,ZHOU Jingyu,WANG Nannan,GAO Xinbo. Cross-modality person re-identification utilizing the hybrid two-stream neural networks[J].Journal of Xidian University, 2021, 48(5): 190-200.

Figures/Tables 9

神经网络层名称	Conv0	Conv1_x	Conv2_x	Conv3_x	Conv4_x
ResNet50 参数说明	7×7,64, stride 2 3×3 max pool, stride 2	$1 × 1,64 3 × 3,64 1 × 1,256$ ×3	$1 × 1,64 3 × 3,64 1 × 1,256$ ×3	$1 × 1,64 3 × 3,64 1 × 1,256$ ×3	$1 × 1,64 3 × 3,64 1 × 1,256$ ×3

References 27

[1]	黄凯奇, 陈晓棠, 康运锋, 等. 智能视频监控技术综述[J]. 计算机学报, 2015, 38(6):1093-1118.
	HUANG Kaiqi, CHEN Xiaotang, KANG Yunfeng, et al. Intelligent Visual Surveillance:A Review[J]. Chinese Journal of Computers, 2015, 38(6):1093-1118.
[2]	叶钰, 王正, 梁超, 等. 多源数据行人重识别研究综述[J]. 自动化学报, 2020, 46(9):1869-1884.
	YE Yu, WANG Zheng, LIANG Chao, et al. A Survey on Multi-Source Person Re-Identification[J]. Acta Automatica Sinica, 2020, 46(9):1869-1884.
[3]	LIAO S, HU Y, ZHU X, et al. Person Re-Identification by Local Maximal Occurrence Representation and Metric Learning[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2015:2197-2206.
[4]	CHENG D, CHANG X, LIU L, et al. Discriminative Dictionary Learning with Ranking Metric Embedded for Person Re-Identification[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence.New York:ACM, 2017:964-970.
[5]	CHENG D, GONG Y, ZHOU S, et al. Person Re-Identification by Multi-Channel Parts-Based CNN with Improved Triplet Loss Function[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:1335-1344.
[6]	LIU J, NI B, YAN Y, et al. Pose Transferrable Person Re-Identification[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:4099-4108.
[7]	SHEN Y, LI H, YI S, et al. Person Re-Identification with Deep Similarity-Guided Graph Neural Network[C]// Proceedings of European Conference on Computer Vision.Heidelberg:Springer, 2018:508-526.
[8]	WEI L, ZHANG S, GAO W, et al. Person Transfer GAN to Bridge Domain Gap for Person Re-Identification[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:79-88.
[9]	CHOPRA S, HADSELL R, LECUN Y. Learning A Similarity Metric Discriminatively,with Application to Face Verification[C]// Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2005:539-546.
[10]	CHEN W, CHEN X, ZHANG J, et al. Beyond Triplet Loss:A Deep Quadruplet Network for Person Re-Identification[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:403-412.
[11]	WANG H, WANG Y, ZHOU Z, et al. CosFace:Large Margin Cosine Loss for Deep Face Recognition[C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:5265-5274.
[12]	ZHENG L, BIE Z, SUN Y, et al. MARS:A Video Benchmark for Large-Scale Person Re-Identification[C]// Proceedings of European Conference on Computer Vision.Heidelberg:Springer, 2016:868-884.
[13]	ZHENG L, SHENG L, TIAN L, et al. Scalable Person Re-Identification:A Benchmark[C]// Proceedings of 2015 IEEE International Conference on Computer Vision.Piscataway:IEEE, 2015:1116-1124.
[14]	WU A, ZHENG W, YU H, et al. RGB-Infrared Cross-Modality Person Re-Identification[C]// Proceedings of 2017 IEEE International Conference on Computer Vision.Piscataway:IEEE, 2017:5390-5399.
[15]	YE Mang, LAN Xiangyuan, LI Jiawei, et al. Hierarchical Discriminative Learning for Visible Thermal Person Re-Identification[C/OL]. [2021-04-30]. https://ojs.aaai.org/index.php/AAAI/outiclel/view/12293.
[16]	HAO Y, WANG N, LI J, et al. HSME:Hypersphere Manifold Embedding for Visible Thermal Person Re-Identification[C]// Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto:AAAI, 2019:8385-8392.
[17]	CHOI S, LEE S, KIM Y, et al. Hi-CMD:Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2020:10257-10266.
[18]	LI Diangang, WEI Xing, HONG Xiaopeng, et al, Infrared-Visible Cross-Modal Person Re-Identification with an X Modality[C/OL]. [2021-04-30]. https://ojs.aaai.org/index.php/AAAI/article/view/5891.
[19]	FU C, HU Y, WU X, et al. CM-NAS:Rethinking Cross-Modality Neural Architectures for Visible-Infrared Person Re-Identification (2021)[J/OL]. [2021-01-21]. https://arxiv.org/abs/2101.08467.
[20]	HAO Y, LI J, WANG N, et al. Dual-Alignment Feature Embedding for Cross-Modality Person Re-Identification[C]// Proceedings of the 27th ACM International Conference on Multimedia.New York:ACM, 2019:57-65.
[21]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:770-778.
[22]	PASZKE A, GROSS S, MASSA F, et al. PyTorch:An Imperative Style,High-Performance Deep Learning Library (2019)[J/OL]. [2019-12-03]. https://arxiv.org/abs/1912.01703v1.
[23]	YE M, LAN X, WANG Z, et al. Bi-Directional Center-Constrained Top-Ranking for Visible Thermal Person Re-Identification[J]. IEEE Transactions on Information Forensics and Security, 2019, 15:407-419. doi: 10.1109/TIFS.10206
[24]	WANG G, ZHANG T, CHENG J, et al. RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision.Piscataway:IEEE, 2019:3622-3631.
[25]	FENG Z, LAI J, XIE X. Learning Modality-Specific Representations for Visible-Infrared Person Re-Identification[J]. IEEE Transactions on Image Processing, 2019, 29:579-590. doi: 10.1109/TIP.83
[26]	YE M, LAN X, LENG Q, et al. Cross-Modality Person Re-Identification via Modality-Aware Collaborative Ensemble Learning[J]. IEEE Transactions on Image Processing, 2020, 29:9387-9399. doi: 10.1109/TIP.83
[27]	WANG Guanan, ZHANG Tianzhu, YANG Yang, et al. Cross Modality Paired-Images Generation for RGB-Infrared Person Re-Identification[C/OL]. [2021-04-30].DOI: 10.1609/aaai.v34io7.6894. doi: 10.1609/aaai.v34io7.6894

参数类型	1≤epoch<10	10≤epoch<20	20≤epoch
骨干网络参数	l_r×epoch×0.1×0.1	l_r×0.1	l_r×0.1×0.1
其他参数	l_r×epoch×0.1	l_r	l_r×0.1

网络结构	Top-1	Top-5	Top-10	Top-20	mAP
全共享^[9]	43.13	N/A	85.37	93.67	43.56
全共享*	47.70	75.41	85.67	93.40	46.10
双通路-0	48.41	77.56	87.33	93.48	47.60
双通路-1	47.76	75.78	85.09	92.66	45.78
双通路-2	49.79	79.31	88.35	94.50	47.83
双通路-3	45.23	75.68	83.64	91.69	44.21
双通路-4	46.83	75.05	84.85	92.90	45.01

优化策略	Top-1	Top-5	Top-10	Top-20	mAP
笔者所提优化策略	49.79	79.31	88.35	94.50	47.83
整个训练过程均使用固定的初始学习率	31.69	59.79	72.15	84.20	31.15
初始训练阶段改为使用固定的初始学习率	45.73	74.68	85.43	92.60	44.97
初始训练阶段改为使用固定的1/10初始学习率	38.86	69.76	80.36	88.98	37.97

方法	Top-1	Top-5	Top-10	Top-20	mAP
Zero-Pad^[14]	14.80	N/A	54.12	71.33	15.95
TONE^[15]	12.52	N/A	50.72	68.60	14.42
BDTR^[23]	17.01	N/A	55.43	71.96	19.66
HSME^[16]	20.68	N/A	62.74	77.95	23.12
AlignGAN^[24]	42.40	N/A	85.00	93.70	40.70
MSR^[25]	37.35	N/A	83.40	93.34	38.11
MACE^[26]	51.64	N/A	87.25	94.44	50.11
JSIA^[27]	38.10	N/A	80.70	89.90	36.90
DFE^[20]	48.71	N/A	88.86	95.27	48.59
DFE*^[20]	49.12	N/A	86.06	92.80	47.92
文中所提方法(双通路-0*)	51.41	77.99	86.83	94.08	49.06
文中所提方法(双通路-2*)	54.22	81.36	88.40	94.53	50.58

方法	可见光—红外
方法	Top-1	Top-10	Top-20	mAP	Top-1	Top-10	Top-20	mAP
Zero-Pad^[14]	17.74	34.21	44.35	18.90	16.63	34.68	44.25	17.82
HCML^[15]	24.44	47.53	56.78	20.08	21.70	45.02	55.58	22.24
BDTR^[23]	33.56	58.61	67.43	32.76	32.92	58.46	68.43	31.96
HSME^[16]	50.85	73.36	81.66	47.00	50.15	72.40	81.07	46.16
AlignGAN^[24]	57.9	N/A	N/A	53.6	56.3	N/A	N/A	53.4
MSR^[25]	48.43	70.32	79.95	48.67	48.43	70.32	79.95	48.67
MACE^[26]	72.37	88.40	93.59	69.09	72.12	88.07	93.07	68.57
JSIA^[27]	48.5	N/A	N/A	49.3	48.1	N/A	N/A	48.9
DFE^[20]	70.13	86.32	91.96	69.14	67.99	85.56	91.41	66.70
双通路-2*	82.54	92.60	95.96	78.80	78.51	90.75	94.51	75.51

Cross-modality person re-identification utilizing the hybrid two-stream neural networks

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 27

Related Articles 1

Metrics

Comments

Recommended 10