西安电子科技大学学报 ›› 2021, Vol. 48 ›› Issue (6): 48-56.doi: 10.19665/j.issn1001-2400.2021.06.007

• 智能嵌入式系统结构与软件关键技术专栏 • 上一篇    下一篇

一种高效的自监督元迁移小样本学习算法

史家辉(),郝小慧(),李雁妮()   

  1. 西安电子科技大学 智能媒体与数据工程研究所,陕西 西安 710071
  • 收稿日期:2021-06-30 出版日期:2021-12-20 发布日期:2022-02-24
  • 通讯作者: 李雁妮
  • 作者简介:史家辉(1997—),男,西安电子科技大学硕士研究生,E-mail: shijh@stu.xidian.edu.cn|郝小慧(1995—),女,西安电子科技大学硕士研究生,E-mail: xhhao@stu.xidian.edu.cn
  • 基金资助:
    国家自然科学基金面上项目(61472296)

Efficient self-supervised meta-transfer algorithm for few shot learning

SHI Jiahui(),HAO Xiaohui(),LI Yanni()   

  1. Institute of Intelligent Media and Data Engineering,Xidian University,Xi’an 710071,China
  • Received:2021-06-30 Online:2021-12-20 Published:2022-02-24
  • Contact: Yanni LI

摘要:

当前深度学习的一个关键难点即小样本问题。尽管已出现了一些较有效的小样本算法,但现有方法的模型提取的特征有限,且模型的泛化能力较弱。另外,如果新类的数据和训练集中数据的分布差异大,分类结果就会很差。针对已有算法的上述缺陷,提出了残差注意力膨胀卷积网络作为网络模型的特征提取器,膨胀分支的设计增大了模型感受野且可以提取不同尺寸的特征,基于图片的残差注意力增强了模型对重要特征的关注度。提出基于自监督的网络模型预训练算法,预训练阶段使用自监督方式,对图像数据进行不同角度旋转且建立相应标签,设计基于图像结构信息的旋转分类器,增加了训练任务中的监督信息,以增强对数据信息进一步挖掘及算法的泛化能力。以目前一些性能最优的小样本算法作为基准性能对比算法,在标准的小样本数据集miniImageNet和Fewshot-CIFAR100上,将文中算法与基准算法进行了充分地实验。实验结果表明:该算法取得了最新最好的性能。

关键词: 小样本, 自监督, 膨胀卷积, 残差注意力

Abstract:

A key difficulty of current deep learning is the problem of few shots.Although some more effective few-shot algorithms/models have appeared,the existing deep models have limited features and the ability of the models to make generalization is low.If the distribution of the data in the new class and that of the data in the training dataset differ greatly,the classification result will be poor.In view of the above-mentioned shortcomings of the existing algorithms,the author proposes the residual attention dilation convolutional network as the feature extractor of the network model.The design of dilation branch increases the model’s receptive field and can extract features of different sizes.Image-based residual attention enhances the model’s attention to important features.A self-supervised network model pre-training algorithm is proposed.The self-supervised method is used in the pre-training stage to rotate the image data at different angles and establish corresponding labels.The rotation classifier based on the image structure information is designed to increase the supervision information in the training task so as to enhance the further mining of data information and the ability of the algorithm to make generalization.On the benchmark few-shot datasets miniImageNet and Fewshot-CIFAR100,the algorithm proposed in this paper is compared with the latest and best few-shot algorithm,with experimental results showing that the algorithm in this paper has achieved the latest and best performance.

Key words: few-shot, self-supervised, dilated convolution, residual attention

中图分类号: 

  • TP183