Electronic Science and Technology ›› 2020, Vol. 33 ›› Issue (3): 12-16.doi: 10.16180/j.cnki.issn1007-7820.2020.03.003

Previous Articles     Next Articles

Multi-label Image Annotation Algorithm Based on Transfer Learning and Weighted Ranking Support Vector Machine

CHEN Lei,LI Feifei,CHEN Qiu   

  1. School of Optical-Electrical and Computer Engineering,University of Shanghai for Science and Technology,Shanghai 200093,China
  • Received:2019-01-24 Online:2020-03-15 Published:2020-03-25
  • Supported by:
    The Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning(ES2012XX);The Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning(ES2014XX)


In order to resolve the class imbalance problem in multi-label image annotation, an improved annotation method based on transfer learning and WRSVM was proposed in this paper. As it was difficult to train a CNN from scratch by using small datasets, transfer learning was adopted to transfer the parameters of Alexnet trained by Imagenet dataset to the convolutional neural network model utilized in the study. Besides, the last fully connected layer was fine-tuned and the multi-label multi-hinge loss function was applied to constitute multi-class support vector machine. Finally, the weighted ranking was used to label the low-frequency labels to obtain the multi-label image annotation results. The experiments were performed on three datasets including Corel-5k, Esp-Game and IAPR-TC12. The experimental results showed that the average recall of the proposed method increased 10%, 9%, and 6%, respectively, and the average of precision increased 12% for the low-frequency labels, indicating the proposed CNN-WRSVM method could improve the average of recall and the average of precision for low-frequency labels.

Key words: multi label image annotation, transfer learning, WRSVM, CNN, MHL, low frequency labels

CLC Number: 

  • TP391