深度共性保持哈希

doi:10.19665/j.issn1001-2400.2021.03.009

摘要/Abstract

摘要：

由于现有的大多数跨模态哈希方法未能有效地探究不同模态数据之间的相关性以及多样性,导致检索性能不尽如人意。为了克服该问题,提出一种简单而有效的深度跨模态哈希方法——深度共性保持哈希,可以在简单的端到端网络中同时学到模态共享表示和模态私有表示,并生成对应模态的判别性紧凑哈希码。与现有的基于深度的跨模态哈希方法相比,所提出的方法的模型复杂度和计算量几乎可以忽略不计,但是获得了显著的性能提升。在三个跨模态数据集上的大量实验结果表明,该方法优于其他当前最先进的跨模态哈希方法。

关键词: 多模态学习, 哈希, 模态共享表示, 模态私有表示, 检索

Abstract:

At present,most existing cross-modal hashing methods fail to explore the relevance and diversity of different modality data,thus leading to unsatisfactory search performance.In order to solve the above problem,a simple yet efficient deep hashing model is proposed,named deep consistency-preserving hashing for cross-modal retrieval that simultaneously exploits modality-common representation and modality-private representation through the simple end-to-end network structure,and generates compact and discriminative hash codes for multiple modalities.Compared with other deep cross-modal hashing methods,the complexity and computation of the proposed method can be neglected with significant performance improvements.Comprehensive evaluations are conducted on three cross-modal benchmark datasets which illustrate that the proposed method is superior to the state-of-the-art cross-modal hashing methods.

Key words: multi-modal learning, hash, modality-common representation, modality-private representation, retrieval

中图分类号:

TP391

石娟,谢德,蒋庆. 深度共性保持哈希[J]. 西安电子科技大学学报, 2021, 48(3): 71-77.

SHI Juan,XIE De,JIANG Qing. Deep consistency-preserving hashing[J]. Journal of Xidian University, 2021, 48(3): 71-77.

图/表 5

图1

表1

图2

表2

图3

参考文献 18

[1]	MA Y, HE Y, DING F, et al. Progressive Generative Hashing for Image Retrieval[C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. Stockholm: AAAI Press, 2018: 871-877.
[2]	FU Q, HAN X, LIU X, et al. Complementary Binary Quantization for Joint Multiple Indexing[C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. Stockholm: AAAI Press, 2018: 2114-2120.
[3]	WANG B, LIU X, XIA K, et al. Random Angular Projection for Fast Nearest Subspace Search[C]// Pacific Rim Conference on Multimedia.Heidelberg:Springer, 2018: 15-26.
[4]	朱莉, 赵俊, 傅应锴, 等. 一种红外热图像目标区域分割的深度学习算法[J]. 西安电子科技大学学报, 2019,46(4):107-114.
	ZHU Li, ZHAO Jun, FU Yingkai, et al. Deep Learning Algorithm for the Segmentation of the Interested Region of an Infrared Thermal Image[J]. Journal of Xidian University, 2019,46(4):107-114.
[5]	蒋行国, 万今朝, 蔡晓东, 等. 奇异值分解与中心度量的细粒度车型识别算法[J]. 西安电子科技大学学报, 2019,46(3):82-88.
	JIANG Xingguo, WAN Jinzhao, CAI Xiaodong, et al. Algorithm for Identification of Fine-grained Vehicles Based on Singular Value Decomposition and Central Metric[J]. Journal of Xidian University, 2019,46(3):82-88.
[6]	刘树东, 王晓敏, 张艳. 一种对称残差CNN的图像超分辨率重建方法[J]. 西安电子科技大学学报, 2019,46(5):15-23.
	LIU Shudong, WANG Xiaomin, ZHANG Yan. Symmetric Residual Convolution Neural Networks for the Image Super Resolution Reconstruction[J]. Journal of Xidian University, 2019,46(5):15-23.
[7]	杨晓莉, 蔺素珍. 一种注意力机制的多波段图像特征级融合方法[J]. 西安电子科技大学学报, 2020,47(1):120-127.
	YANG Xiaoli, LIN Suzhen. Method for Multi-Band Image Feature-Level Fusion Based on the Attention Mechanism[J]. Journal of Xidian University, 2020,47(1):120-127.
[8]	LIONG V E, LU J, TAN Y, et al. Cross-Modal Deep Variational Hashing[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision(ICCV).Piscataway:IEEE, 2017: 4097-4105.
[9]	JIANG Q and LI W. Deep Cross-Modal Hashing[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017: 3270-3278.
[10]	DENG C, YANG E, LIU T, et al. Unsupervised Semantic-Preserving Adversarial Hashing for Image Search[J]. IEEE Transactions on Image Processing, 2019,28(8):4032-4044. doi: 10.1109/TIP.83
[11]	LI C, DENG C, LI N, et al. Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval[C]// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018: 4242-4251.
[12]	BOUSMALIS K, TRIGEORGIS G, SILBERMAN N, et al. Domain Separation Networks[C]// Advances in Neural Information Processing Systems. Barcelona: MIT Press, 2016: 343-351.
[13]	ZHANG X, LAI H, FENG J S, et al. Attention-Aware Deep Adversarial Hashing for Cross-Modal Retrieval[C]// Proceedings of the European Conference on Computer Vision.Munich:Springer, 2018: 591-606.
[14]	KUMAR S, UDUPA R. Learning Hash Functions for Cross-View Similarity Search[C]// Twenty-Second International Joint Conference on Artificial Intelligence.Barcelona:Morgan Kaufmann, 2011: 1360-1365.
[15]	WANG D, GAO X B, WANG X M, et al. Semantic Topic Multimodal Hashing for Cross-Media Retrieval[C]// Twenty-Fourth International Joint Conference on Artificial Intelligence.Buenos Aires:Morgan Kaufmann, 2015: 3890-3896.
[16]	BRONSTEIN M M, BRONSTEIN A M, MICHEL F, et al. Data Fusion Through Cross-Modality Metric Learning Using Similarity-Sensitive Hashing[C]// Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco:ZEEE, 2010: 3594-3601.
[17]	ZHANG D Q, LI W J. Large-Scale Supervised Multimodal Hashing with Semantic Correlation Maximization[C]// Twenty-Eighth AAAI Conference on Artificial Intelligence.Quebec:AAAI, 2014: 2177-2183.
[18]	ZHOU J L, DING G G. Latent Semantic Sparse Hashing for Cross-Modal Similarity Search[C]// Proceedings of the 37th international ACM SIGIR Conference on Research & Development in Information Retrieval.Gold Coast:ACM, 2014: 415-424.

任务	方法	MIRFLICKR-25K			IAPRTC-12			NUS-WIDE
任务	方法	16 bit	32 bit	64 bit	16 bit	32 bit	64 bit	16 bit	32 bit	64 bit
I→T	CVH^[14]	0.587	0.586	0.581	0.444	0.465	0.47	0.508	0.508	0.508
	STMH^[15]	0.721	0.729	0.774	0.463	0.540	0.578	0.689	0.763	0.809
	CMSSH^[16]	0.64	0.645	0.645	0.509	0.462	0.492	0.671	0.698	0.578
	SCM^[17]	0.778	0.791	0.797	0.651	0.667	0.677	0.645	0.677	0.691
	LSSH^[18]	0.647	0.714	0.713	0.562	0.592	0.603	0.611	0.647	0.741
	DCMH^[9]	0.834	0.848	0.855	0.662	0.682	0.721	0.567	0.606	0.646
	文中方法	0.903	0.900	0.907	0.857	0.839	0.871	0.752	0.821	0.847
T→I	CVH^[14]	0.504	0.517	0.525	0.362	0.285	0.299	0.261	0.256	0.252
	STMH^[15]	0.671	0.697	0.725	0.459	0.510	0.573	0.603	0.679	0.702
	CMSSH^[16]	0.548	0.486	0.504	0.379	0.411	0.375	0.541	0.448	0.469
	SCM^[17]	0.812	0.818	0.819	0.720	0.744	0.754	0.707	0.740	0.755
	LSSH^[18]	0.654	0.691	0.719	0.502	0.553	0.583	0.554	0.628	0.691
	DCMH^[9]	0.840	0.860	0.861	0.717	0.692	0.78	0.662	0.672	0.700
	文中方法	0.874	0.880	0.891	0.845	0.844	0.873	0.769	0.832	0.852

任务	方法	比特数
任务	方法	16 bit	32 bit	64 bit
I→T	CMDVH^[8]	0.720	0.773	0.800
	SSAH^[11]	0.790	0.797	0.817
	文中方法	0.842	0.864	0.878
T→I	CMDVH^[8]	0.735	0.774	0.804
	SSAH^[11]	0.786	0.819	0.808
	文中方法	0.813	0.847	0.852