基于迁移学习与权重支持向量机的图像多标签标注算法

doi:10.16180/j.cnki.issn1007-7820.2020.03.003

Abstract

Abstract:

In order to resolve the class imbalance problem in multi-label image annotation, an improved annotation method based on transfer learning and WRSVM was proposed in this paper. As it was difficult to train a CNN from scratch by using small datasets, transfer learning was adopted to transfer the parameters of Alexnet trained by Imagenet dataset to the convolutional neural network model utilized in the study. Besides, the last fully connected layer was fine-tuned and the multi-label multi-hinge loss function was applied to constitute multi-class support vector machine. Finally, the weighted ranking was used to label the low-frequency labels to obtain the multi-label image annotation results. The experiments were performed on three datasets including Corel-5k, Esp-Game and IAPR-TC12. The experimental results showed that the average recall of the proposed method increased 10%, 9%, and 6%, respectively, and the average of precision increased 12% for the low-frequency labels, indicating the proposed CNN-WRSVM method could improve the average of recall and the average of precision for low-frequency labels.

Key words: multi label image annotation, transfer learning, WRSVM, CNN, MHL, low frequency labels

CLC Number:

TP391

CHEN Lei,LI Feifei,CHEN Qiu. Multi-label Image Annotation Algorithm Based on Transfer Learning and Weighted Ranking Support Vector Machine[J].Electronic Science and Technology, 2020, 33(3): 12-16.

Figures/Tables 5

Figure 1.

Table 1

Table 2

Table 3

Table 4

References 17

[1]	秦莹华, 李菲菲, 陈虬 . 基于迁移学习的多标签图像标注[J]. 电子科技, 2018,31(8):25-28.
	Qing Yinghua, Li Feifei, Chen Qiu . Multi-label image annotation based on transfer learning[J]. Electronic Science and Technology, 2018,31(8):25-28.
[2]	Yang X, Qian X, Mei T . Learning salient visual word for scalable mobile image retrieval[J]. Pattern Recognition, 2015,48(2):3093-3101.
[3]	Cheng Q, Zhang Q, Fu P , et al. A survey and analysis on automatic image annotation[J]. Pattern Recognition, 2018,79(3):242-259.
[4]	Zhang R, Zhang L, Wang X J, et al. Multi-feature pLSA for combining visual features in image annotation[C]. Scottsdale:International Conference on Multimedia, 2011.
[5]	Kalayeh M M, Idrees H, Shah M . NMF-KNN:image annotation using weighted multi-view non-negative matrix factorization[C]. Columbus:IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
[6]	Verma, Yashaswi, Jawahar C V. Image annotation using metric learning in semantic neighbourhoods[C]. Berlin:European Conference on Computer Vision, 2012.
[7]	Hou Y, Lin Z. Image tag completion and refinement by subspace clustering and matrix completion[C]. Chengdu:Visual Communications & Image Processing, 2016.
[8]	Lin Z, Ding G, Hu M , et al. Image tag completion via dual-view linear sparse reconstructions[J]. Computer Vision and Image Understanding, 2014,124(6):42-60.
[9]	Jing X Y, Wu F, Li Z , et al. Multi-label dictionary learning for image annotation[J]. IEEE Transactions on Image Processing, 2016,25(6):2712-2725.
[10]	Murthy V N, Can E F, Manmatha R. A hybrid mod-el for automatic image annotation[C]. Glasgow:Proceedings of International Conference on Multimedia Retrieval, 2014.
[11]	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural net-works[C]. Stateline:International Conference on Neural Information Processing Systems, 2012.
[12]	Pan S J, Yang Q . A survey on transfer learning[J]. IEEE Transactions on Knowledge & Data Engineering, 2010,22(10):1345-1359.
[13]	Yosinski J, Clune J, Bengio Y, et al. How transfera ble are features in deep neural networks[C]. Montréal: Advances in Neural Information Processing Systems, 2014.
[14]	Chen M, Zheng A, Weinberger K. Fast image tag ging[C]. Atlanta:International Conference on Machine Learning, 2013.
[15]	Murthy V N, Maji S, Manmatha R. Automatic image annotation using deep learning representations[C]. Shanghai:Proceedings of the 5 ^th ACM on International Conference on Multimedia Retrieval , 2015.
[16]	汪鹏, 张奥帆, 王利琴 , 等. 基于迁移学习与多标签平滑策略的图像自动标注[J]. 计算机应用, 2018,38(11):3199-3203.
	Wang Peng, Zhang Aofan, Wang Liqim , et al. Image au-tomatic annotation based on transfer learning and multi-label smoothing strategy[J]. Journal of Computer Applications, 2018,38(11):3199-3203.
[17]	Kashani M M, Amiri S H. Leveraging deep learn-ing representation for search-based image annotation[C]. Shiraz:Artificial Intelligence & Signal Processing Conference, 2018.

层级名称	设置参数
输入层	图像尺寸为256×256
卷积层1	卷积核尺寸为11×11,最大池化层
卷积层2	卷积核尺寸为5×5,最大池化层
卷积层3	卷积核尺寸为3×3
卷积层4	卷积核尺寸为3×3
卷积层5	卷积核尺寸为3×3,最大池化层
全连接层1	输出特征向量维数4 096
全连接层2	输出特征向量维数4 096
全连接层3	输出特征向量维数4 096
输出层	多标签损失层,输出向量维数依数据集标签个数决定

数据集	Corel-5k	Esp-Game	IAPR-TC12
图像总数	5 000	20 770	19 627
训练集图像数	4 500	18 689	17 665
测试集图像数	499	2 081	1 962
标签数	260	268	291
平均标签数/幅	3.4	4.7	5.7
α	58.6	362.7	347.7

数据集	Corel-5k				ESP-Game				IAPR-TC 12
数据集	P	R	F₁	N+	P	R	F₁	N+	P	R	F₁	N+
2PKNN-ML^[6]	0.44	0.46	0.45	191	0.53	0.27	0.35	252	0.54	0.37	0.43	278
FastTag^[14]	0.32	0.43	0.37	166	0.46	0.22	0.30	247	0.47	0.26	0.34	280
CCA-KNN^[15]	0.42	0.52	0.46	201	0.46	0.36	0.41	260	0.45	0.38	0.41	278
MLDL^[9]	0.45	0.49	0.47	198	0.56	0.31	0.40	259	0.56	0.40	0.47	282
CNN-MLSU^[16]	0.37	0.49	0.42	-	-	-	-	-	0.44	0.38	0.41	-
NN-CNN^[17]	0.42	0.45	0.43	187	0.50	0.28	0.36	252	0.54	0.31	0.40	272
CNN-WRSVM	0.44	0.62	0.51	137	0.42	0.45	0.44	222	0.48	0.42	0.45	222

数据集	Corel-5k				ESP-Game				IAPR-TC 12
数据集	P	R	F₁	N+	P	R	F₁	N+	P	R	F₁	N+
CNN-SVM	0.25	0.39	0.31	91	0.43	0.31	0.36	188	0.50	0.30	0.38	166
CNN-WRSVM	0.37	0.32	0.34	72	0.55	0.25	0.35	163	0.32	0.25	0.38	148

Multi-label Image Annotation Algorithm Based on Transfer Learning and Weighted Ranking Support Vector Machine

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 17

Related Articles 13

Metrics

Comments

Recommended 10

[1]	CHENG Xiaoya,ZHANG Lei. Research on Occluded Face Recognition Method Based on Deep Learning [J]. Electronic Science and Technology, 2022, 35(1): 35-39.
[2]	MA Lixin,DOU Chenfei,SONG Chencan,YANG Tianxiao. Insulator Nondestructive Testing Based on Feature Fusion CNN [J]. Electronic Science and Technology, 2021, 34(7): 26-30.
[3]	ZHAI Yuexian,LIU Xiang,SONG Jialin. Staging Diagnosis of Liver Fibrosis Based on Transfer Learning Strategy [J]. Electronic Science and Technology, 2021, 34(6): 11-16.
[4]	YAN Shuhao,QIAO Meiying. Bearing Fault Diagnosis Algorithm Based on One-Dimensional WConv-BiLSTM [J]. Electronic Science and Technology, 2021, 34(4): 75-82.
[5]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.
[6]	GE Jing,LIU Zilong. The Algorithm Based on CNN and LSTM for Sleep Apnea Syndrome Detection [J]. Electronic Science and Technology, 2021, 34(2): 21-26.
[7]	ZHANG Ying,LIU Zilong,WAN Wei. UAV Vehicle Target Detection Based on Faster R-CNN [J]. Electronic Science and Technology, 2021, 34(11): 11-20.
[8]	SU Ye,LI Jun,YANG Haima,LIU Jin,JIANG Shenghua. Research on Rail Surface Defect Recognition Based on Faster R-CNN [J]. Electronic Science and Technology, 2020, 33(9): 63-68.
[9]	WANG Zhengjun,YAO Yiming,CHEN Long. PCB Board Common Sorting Algorithm Based on Convolutional Neural Network [J]. Electronic Science and Technology, 2020, 33(2): 60-65.
[10]	ZHOU Wenkai,HAN Fang,KONG Weijian. Point-selective Geetest CAPTCHA Recognition Based on Faster-RCNN [J]. Electronic Science and Technology, 2019, 32(9): 42-46.
[11]	LI Rongrui,SHI Lin,ZHAO Wei. Minority Headdress Recognition Based on Convolutional Neural Network [J]. Electronic Science and Technology, 2019, 32(2): 51-55.
[12]	YUAN Xiaoping,WANG Gang,WANG Yefeng,WANG Zheyuan,SUN Hui. Traffic Sign Recognition Method Based on Improved Convolutional Neural Network [J]. Electronic Science and Technology, 2019, 32(11): 28-32.
[13]	MA Lixin,ZHOU Xiaobo,SHAN Yu. UV Image Fusion Based on Adaptive PCNN [J]. , 2015, 28(10): 1-.