判别与结构信息保持的无监督领域自适应方法

doi:10.19665/j.issn1001-2400.2022.04.011

摘要/Abstract

摘要：

基于特征表示的领域自适应是研究的最广泛的迁移学习方法。但是,目前的方法在获得新的数据表征之后,没有考虑类内与内间的关系,并且在转换的过程中丢失了数据的几何结构信息。针对上述问题,提出一种判别与结构信息保持的无监督领域自适应方法。该方法在低秩子空间学习的框架下,从保留源域的判别信息和数据的结构信息两方面考虑,提高分类器在目标域样本的分类准确度。具体地,该方法寻找一个源域和目标域之间的不变子空间,通过低秩约束的跨领域样本重构学习以减少跨域分布差异,在迁移过程中,使用标签松弛矩阵最大化源域不同类别样本的类间距离和源域类间稀疏约束缩小来自同类别样本的距离,有效地保留了源域的判别信息。同时,利用自适应概率图结构保留样本的局部近邻关系,捕获数据底层的几何结构信息,增强子空间学习的鉴别力和鲁棒性。在3种不同的跨域图像基准数据集上实验验证了提出方法的有效性。实验结果表明,所提算法的分类性能优于目前存在的方法。

关键词: 迁移学习, 子空间学习, 领域自适应, 类间稀疏, 图结构

Abstract:

One popular transfer learning method is domain adaptation based on feature representation.However,such a method fails to consider the within-class and between-class relations after obtaining the new data representation.In addition,the geometric structure information of the data is lost during the conversion process.To overcome this problem,a novel discrimination and structure preserved cross-domain subspace learning for the unsupervised domain adaption method is developed in this paper.Under the framework of low rank subspace learning,this method improves the classification accuracy of the classifier in the target domain by considering both the discrimination information of the source domain and the structural information of the data.Specifically,the method finds an invariant subspace between the source domain and the target domain,and uses cross domain sample reconstruction learning with low rank constraints to reduce the difference of cross domain distribution.In the process of migration,the label relaxation matrix is used to maximize the inter class distance of samples of different categories in the source domain and the sparse constraint between classes in the source domain to reduce the distance from samples of the same category and effectively retain the discrimination information of the source domain.At the same time,the adaptive probability graph structure is used to retain the local nearest neighbor relationship of samples,capture the geometric structure information at the bottom of the data,and enhance the discrimination and robustness of subspace learning.Experiments on three different cross-domain image data sets verify the effectiveness of the proposed method.Experimental results show that the classification performance of the proposed algorithm is better than that of the existing methods.

Key words: transfer learning, subspace learning, domain adaptation, Inter-class sparsity, graph structure

中图分类号:

TP181

陶洋,杨娜,郭坦. 判别与结构信息保持的无监督领域自适应方法[J]. 西安电子科技大学学报, 2022, 49(4): 90-99.

TAO Yang,YANG Na,GUO Tan. Discrimination and structure preserved cross-domain subspace learning for unsupervised domain adaption[J]. Journal of Xidian University, 2022, 49(4): 90-99.

图/表 10

图1

图2

表1

表2

表3

表4

表5

图3

图4

表6

参考文献 23

[1]	PANIGRAHI S, NANDA A, SWARNKAR T. A Survey on Transfer Learning[C]// Intelligent and Cloud Computing.Heidelberg:Springer, 2021:781-789.
[2]	AGARWAL N, SONDHI A, CHOPRA K, et al. Transfer Learning:Survey and Classification[C]// Smart Innovations in Communication and Computational Sciences.Heidelberg:Springer, 2021:145-155.
[3]	XIE Y, DU Z, LI J, et al. Joint Metric and Feature Representation Learning for Unsupervised Domain Adaptation[J]. Knowledge-Based Systems, 2020, 192:105222. doi: 10.1016/j.knosys.2019.105222
[4]	ZHANG L. Transfer Adaptation Learning:A Decade Survey (2019)[J/OL]. [2019-03-12]. https://arxiv.org/abs/1903.04687v1.
[5]	YANG LR, ZHONG P. Discriminative and Informative Joint Distribution Adaptation for Unsupervised Domain Adaptation[J]. Knowledge-Based Systems, 2020, 207:106394. doi: 10.1016/j.knosys.2020.106394
[6]	ZHANG L, FU J, WANG S, et al. Guide Subspace Learning for Unsupervised Domain Adaptation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 31(9):3374-3388. doi: 10.1109/TNNLS.2019.2944455
[7]	FU J R, ZHANG L. Reliable Domain Adaptation with Classifiers Competition for Image Classification[J]. IEEE Transactions on Circuits and Systems II:Express Briefs, 2020, 67(11):2802-2806. doi: 10.1109/TCSII.2020.2977039
[8]	逯彦, 廖桂生, 黄庆享. 结合稀疏表示的跨摄像头运动目标跟踪算法[J]. 西安电子科技大学学报, 2021, 48(2):197-204.
	LU Yan, LIAO Guisheng, HUANG Qingxiang. Cross-Camera Moving Target Tracking Algorithm Based on Sparse Representation[J]. Journal of Xidian University, 2021, 48(2):197-204.
[9]	SHAO M, CASTILLO C, GU Z H, et al. Low-Rank Transfer Subspace Learning[C]// 2012 IEEE 12th International Conference on Data Mining.Piscataway:IEEE, 2012:1104-1109.
[10]	XU Y, FANG X Z, WU J, et al. Discriminative Transfer Subspace Learning via Low-Rank and Sparse Representation[J]. IEEE Transactions on Image Processing, 2015, 25(2):850-863. doi: 10.1109/TIP.2015.2510498
[11]	HE X F, DENG C, YAN S C, et al. Neighborhood Preserving Embedding[C]// Tenth IEEE International Conference on Computer Vision.Piscataway:IEEE, 2005:1-6.
[12]	FANG X Z, XU Y, LI X L, et al. Regularized Label Relaxation Linear Regression[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 29(4):1006-1018. doi: 10.1109/TNNLS.2017.2648880
[13]	CAI X, NIE F P, HUANG H. Exact Top-K Feature Selection via l2,0-Norm Constraint[C]// Twenty-third International Joint Conference on Artificial Intelligence. New York: ACM, 2013:1240-1246.
[14]	ZHANG Z, SHAO L, XU Y, et al. Marginal Representation Learning with Graph Structure Self-Adaptation[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 29(10):4645-4659. doi: 10.1109/TNNLS.2017.2772264
[15]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet Classification with Deep Convolutional Neural Networks[J]. Advances in Neural Information Processing Systems, 2012, 25:1097-1105.
[16]	GONG B, SHI Y, SHA F, et al. Geodesic Flow Kernel for Unsupervised Domain Adaptation[C]// 2012 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2012:2066-2073.
[17]	SHAO M, KIT D, FU Y. Generalized Transfer Subspace Learning through Low-Rank Constraint[J]. International Journal of Computer Vision, 2014, 109(1-2):74-93. doi: 10.1007/s11263-014-0696-6
[18]	LONG M S, WANG J M, DING GG, et al. Transfer Feature Learning with Joint Distribution Adaptation[C]// Proceedings of the IEEE International Conference on Computer Vision.Piscataway:IEEE, 2013:2200-2207.
[19]	WANG J, CHEN Y, HAO S, et al. Balanced Distribution Adaptation for Transfer Learning[C]// 2017 IEEE International Conference on Data Mining (ICDM).Piscataway:IEEE, 2017:1129-1134.
[20]	RAZZAGHI P, RAZZAGHI P, ABBASI K. Transfer Subspace Learning via Low-Rank and Discriminative Reconstruction Matrix[J]. Knowledge-Based Systems, 2019, 163:174-185. doi: 10.1016/j.knosys.2018.08.026
[21]	XIAO T, LIU P, ZHAO W, et al. Structure Preservation and Distribution Alignment in Discriminative Transfer Subspace Learning[J]. Neurocomputing, 2019, 337:218-234. doi: 10.1016/j.neucom.2019.01.069
[22]	TZENG E, HOFFMAN J, ZHANG N, et al. Deep Domain Confusion:Maximizing for Domain Invariance (2014)[J/OL]. [2014-12-10]. https://arxiv.org/abs/1412.3474.
[23]	LONG M, CAO Y, WANG J, et al. Learning Transferable Features with Deep Adaptation Networks[C]// International Conference on Machine Learning. New York: ACM, 2015:97-105.

任务	NN	LTSL	GFK	JDA	LRSR	BDA	SPDA	文中方法
C₁→C₂	83.61	75.69	72.50	89.31	88.61	97.22	97.92	97.92
C₂→C₁	82.78	72.22	74.17	88.47	89.17	96.81	98.89	98.61
平均	83.20	73.69	73.34	88.89	88.89	97.02	98.41	98.27

任务	NN	LTSL	GFK	JDA	LRSR	LRDRM	文中方法
M→V	28.63	26.27	28.76	28.43	34.51	35.03	32.48
V→M	48.94	56.34	48.86	48.94	53.28	54.53	57.53
平均	38.79	41.31	38.81	38.69	44.16	44.78	45.01

任务	NN	LTSL	GFK	JDA	LRSR	BDA	LRDRM	SPDA	文中方法
C→A	23.70	50.57	41.02	44.78	51.25	44.89	53.91	52.82	58.04
C→W	25.76	47.15	40.68	41.69	38.64	38.64	40.00	40.68	55.59
C→D	25.48	48.40	38.85	45.22	47.13	47.77	49.68	51.59	50.96
A→C	26.00	37.65	40.25	39.36	43.37	40.78	44.07	43.37	45.24
A→W	29.83	39.03	38.98	37.97	36.61	39.32	37.28	43.39	48.14
A→D	25.48	38.89	36.31	39.49	38.85	43.31	42.03	46.50	47.13
W→C	19.86	35.45	30.72	31.17	29.83	28.94	35.12	31.97	38.11
W→A	22.96	45.17	29.75	32.78	34.13	32.99	35.07	37.27	41.75
W→D	59.24	72.61	80.89	89.17	82.80	91.72	84.71	89.81	89.17
D→C	26.27	35.08	30.28	31.52	31.61	32.50	36.42	33.84	38.82
D→A	28.50	38.00	32.05	33.09	33.19	33.09	35.07	38.20	42.90
D→W	63.39	73.75	75.59	89.49	77.29	91.86	76.94	82.37	89.15
平均	31.37	46.83	42.95	46.31	45.39	47.15	47.45	49.32	53.75

任务	NN	GFK	JDA	LRSR	BDA	SPDA	AlexNet*	DDC*	DNA*	文中方法
C→A	23.70	87.27	90.19	91.54	89.87	92.90	91.90	91.90	92.00	93.11
C→W	25.76	75.93	85.42	80.00	85.42	92.54	83.70	85.40	90.60	93.56
C→D	25.48	83.44	85.99	87.90	83.44	95.54	87.10	88.80	89.30	92.36
A→C	26.00	80.32	81.92	85.93	81.66	88.25	83.00	85.00	84.10	87.98
A→W	29.83	76.95	80.68	75.25	79.32	89.15	79.50	86.10	91.80	92.54
A→D	25.48	80.89	81.53	84.08	81.53	84.08	87.40	89.00	91.70	93.63
W→C	19.86	67.76	81.21	72.93	82.90	86.11	73.00	78.00	81.20	87.18
W→A	22.96	74.32	90.71	76.83	90.19	93.22	83.80	84.90	92.10	94.26
W→D	59.24	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0	100.0
D→C	26.27	69.10	80.32	87.44	85.31	87.62	79.00	81.10	80.30	88.87
D→A	28.50	75.78	91.96	93.42	91.65	93.53	87.10	89.50	90.00	94.36
D→W	63.39	98.64	99.32	99.32	99.66	100.0	97.70	98.20	98.50	100.0
平均	31.37	80.87	87.44	86.22	87.56	91.91	86.10	88.16	90.13	93.15

任务	C→A	C→W	C→D	A→C	A→W	A→D	W→C	W→A	W→D	D→C	D→A	D→W	平均
原始	90.71	81.69	85.35	81.83	80.00	85.35	84.86	91.44	100.0	86.82	89.25	98.98	88.02
No-GR	92.90	87.80	90.45	86.29	83.73	91.08	84.95	92.38	100.0	87.18	92.38	99.66	90.73
No-ICS	92.69	89.15	90.45	86.36	89.51	92.36	86.91	94.26	100.0	88.51	93.42	99.66	91.94
本文	93.11	93.56	92.36	87.98	92.54	93.63	87.18	94.26	100.0	88.87	94.36	100.0	93.15