基于深度迁移学习与多尺度特征融合的场景识别方法

doi:10.16180/j.cnki.issn1007-7820.2023.11.004

Abstract

Abstract:

CNN(Convolutional Neural Networks) hase achieved excellent results in the field of scene recognition research, but this method do not fully take into account the particularity of the scene. Due to different scales, viewpoints, and backgrounds, there exists large intra-class variation within the same scene class. On the other hand, the common objects also result in a certain inter-class similarities among heterogeneous scenes as well. Considering that scene images of different scales will affect the size of objects in them, this study proposes a scene recognition algorithm based on deep transfer learning and multi-scale feature fusion. First, the network parameters pre-trained on the Places data set are migrated to the CNN model used in this study using migration learning, and then the network is fine-tuned and retrained to reduce the training cost. Secondly, the multi-scale image blocks obtained from the class activation map are fed into the CNN for feature extraction, and the obtained feature vectors are fused to make the final scene image features more comprehensive. Experiment results carried out on the SUN397 data set show that compared with other CNN-based algorithms, the proposed algorithm significantly improves the accuracy of scene recognition.

Key words: scene recognition, convolutional neural network, SE-Block, class activation map, transfer learning, multi-scale, feature fusion, support vector machine

CLC Number:

TP391

WANG Qiao,HU Chunyan,LI Feifei. Scene Recognition Algorithm Based on Deep Transfer Learning and Multi-Scale Feature Fusion[J].Electronic Science and Technology, 2023, 36(11): 19-27.

Figures/Tables 13

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Table 1.

Figure 9.

Figure 10.

Table 2.

Table 3.

References 26

[1]	Le Cun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324. doi: 10.1109/5.726791
[2]	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90. doi: 10.1145/3065386
[3]	Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]. Boston: IEEE Conference on Computer Vision and Pattern Recognition, 2015:1-9.
[4]	Muhammad U, Wang W, Chattha S P, et al. Pre-trained VGGNet architecture for remote-sensing image scene classification[C]. Beijing: The Twenty-fourth International Conference on Pattern Recognition, 2018:1622-1627.
[5]	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]. Las Vegas: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016:770-778.
[6]	Herranz L, Jiang S, Li X. Scene recognition with CNNs:objects,scales and dataset bias[C]. Las Vegas: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016:571-579.
[7]	Chen L, Bo K, Lee F, et al. Advanced feature fusion algorithm based on multiple convolutional neural network for scene recognition[J]. Computer Modeling in Engineering & Sciences, 2020, 122(2):505-523.
[8]	Zhao Z, Larson M. From volcano to toyshop:Adaptive discriminative region discovery for scene recognition[C]. New York: Proceedings of the Twenty-sixth ACM International Conference on Multimedia, 2018:1760-1768.
[9]	Liu Y, Chen Q, Chen W, et al. Dictionary learning inspired deep network for scene recognition[C]. New Orleans: The Thirty-second AAAI Conference on Artificial Intelligence, 2018:7178-7185.
[10]	谢林, 李菲菲, 陈虬. 基于稀疏自动编码机的场景识别算法[J]. 电子科技, 2019, 32(1):38-41.
	Xie Lin, Li Feifei, Chen Qiu. Scene recognition algorithm based on sparse autoencoder[J]. Electronic Science and Technology, 2019, 32(1):38-41.
[11]	缪冉, 李菲菲, 陈虬. 基于卷积神经网络与多尺度空间编码的场景识别方法[J]. 电子科技, 2020, 33(12):1-7.
	Miao Ran, Li Feifei, Chen Qiu. Scene recognition algorithm based on convolutional neural networks and multi-scale space encoding[J]. Electronic Science and Technology, 2020, 33(12):1-7.
[12]	Cheng X, Lu J, Feng J, et al. Scene recognition with objectness[J]. Pattern Recognition, 2018, 74(9):474-487. doi: 10.1016/j.patcog.2017.09.025
[13]	Pan Y, Xia Y, Shen D. Foreground fisher vector:Encoding class-relevant foreground to improve image classification[J]. IEEE Transactions on Image Processing, 2019, 28(10):4716-4729. doi: 10.1109/TIP.83
[14]	Jégou H, Douze M, Schmid C, et al. Aggregating local descriptors into a compact image representation[C]. San Francisco: Computer Vision and Pattern Recognition, 2010:3304-3311.
[15]	Wang Z, Wang L, Wang Y, et al. Weakly supervised patchnets:Describing and aggregating local patches for scene recognition[J]. IEEE Transactions on Image Processing, 2017, 26(4):2028-2041. doi: 10.1109/TIP.2017.2666739 pmid: 28207394
[16]	Pan S J, Yang Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2009, 22(10):345-1359.
[17]	Zhuang F, Qi Z, Duan K, et al. A comprehensive survey on transfer learning[J]. Proceedings of the IEEE, 2020, 109(1):43-76. doi: 10.1109/PROC.5
[18]	李新叶, 龙慎鹏, 朱婧. 基于深度神经网络的少样本学习综述[J]. 计算机应用研究, 2020, 37(8):2241-2247.
	Li Xinye, Long Shenpeng, Zhu Jing. Survey of few-shot learning based on deep neural network[J]. Application Research of Computers, 2020, 37(8):2241-2247.
[19]	Yang S, Lee F F, Miao R. et al. RS-CapsNet:An advanced capsule network[J]. IEEE Access, 2020, 8(10):85007-85018. doi: 10.1109/Access.6287639
[20]	Li T W, Lee G C. Performance analysis of fine-tune transferred deep learning[C]. Yunlin: IEEE the Third Eurasia Conference on IOT,Communication and Engineering, 2021:315-319.
[21]	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]. Las Vegas: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016:770-778.
[22]	王宪保, 肖本督, 姚明海. 一种结合类激活映射的半监督图像分类方法[J]. 小型微型计算机系统, 2022, 43(6):1204-1209.
	Wang Xianbao, Xiao Bendu, Yao Minghai. Semi-supervised image classification method combined with class activation mapping[J]. Journal of Chinese Computer Systems, 2022, 43(6):1204-1209.
[23]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Salt Lake City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:7132-7141.
[24]	Xiao J, Hays J, Ehinger K A, et al. Sun database:Large- scale scene recognition from abbey to zoo[C]. San Francisco: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2010:3485-3492.
[25]	朱晓慧, 钱丽萍, 傅伟. 图像数据增强技术研究综述[J]. 软件导刊, 2021, 20(5):230-236.
	Zhu Xiaohui, Qian Liping, Fu Wei. Overview of research on image data enhancement technology[J]. Software Guide, 2021, 20(5):230-236.
[26]	Shi J, Zhu H, Yu S, et al. Scene categorization model using deep visually sensitive features[J]. IEEE Access, 2019, 7(7):45230-45239. doi: 10.1109/Access.6287639

	批大小	迭代次数	学习率	学习衰减率	动量	权重衰减
特征提取器的分类器	50	40	1×10^-2	10	0.9	-
特征提取器	128	40	1×10^-2	20	0.9	-
类激活图生成	64	100	1×10^-5 (卷积层) 1×10^-2 (其余层)	20	0.9	1×10^-3

序号	类激活图生成器主干网络	特征提取器主干网络			准确度 /%
序号	类激活图生成器主干网络	全局尺度	1/4尺度	1/16尺度	准确度 /%
1	Places	Places	Places	ImageNet	73.93
2	Places/SE/ Finetune	Places/ Finetune	Places/ Finetune	ImageNet	76.18

算法	准确度/%
VS-CNN^[26]	43.14
Dual CNN-DL^[9]	70.13
Multi-scale CNNs^[6]	70.17
Adi-Red^[8]	73.59
本文方法	76.11

Scene Recognition Algorithm Based on Deep Transfer Learning and Multi-Scale Feature Fusion

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 26

Related Articles 15

Metrics

Comments

Recommended 10

[1]	Bin ,WANG Sen. Visual Detection of Structural Cracks Using Depth Deformable Contour ModelLAI [J]. Electronic Science and Technology, 2023, 36(9): 35-40.
[2]	YUE Shengyao,XU Baiqiang,XU Guidong,XU Chenguang,ZHANG Sai. Super-Resolution Imaging of Laminate Debonding Defects via Deconvolutional Neural Network and Ultrasound Guided Waves [J]. Electronic Science and Technology, 2023, 36(8): 7-13.
[3]	ZHA Junwei,ZHANG Hongyan. Dynamic Receptive Field Feature Selection Dehazing Network [J]. Electronic Science and Technology, 2023, 36(7): 56-63.
[4]	SUN Hong,ZHAO Yingzhi. Lightweight Generative Adversarial Networks Based on Multi-Scale Gradient [J]. Electronic Science and Technology, 2023, 36(7): 32-38.
[5]	OU Jingyi,TIAN Ying,XIANG Xin,SONG Qizhe. Fault Diagnosis of Few Shot Industrial Process Based on Transfer BN-CNN Framework [J]. Electronic Science and Technology, 2023, 36(7): 49-55.
[6]	ZENG Xinxin,ZHANG Hongyan. A Farmland Parcel Extraction Network Based on Multi-Scale Semantic Information Enhancement [J]. Electronic Science and Technology, 2023, 36(7): 70-74.
[7]	SHI Jianke,QIAO Meiying,LI Bingfeng,ZHAO Yan. Underwater Occlusion Target Detection Algorithm Based on Attention Mechanism [J]. Electronic Science and Technology, 2023, 36(5): 62-70.
[8]	CUI Zhuodong,CHEN Wei,YIN Zhong. Helmet Wearing Detection Based on Enhanced Feature Fusion Network [J]. Electronic Science and Technology, 2023, 36(4): 44-51.
[9]	SUN Hong,ZHANG Yuxiang. Super-Resolution Image Reconstruction Algorithm Based on Multi-Feature Gated Feedback Residual Network [J]. Electronic Science and Technology, 2023, 36(4): 65-70.
[10]	HUANG Yuan,WEI Yunbing,TONG Dongbing,WANG Weigao. Short-Term Photovoltaic Power Prediction Based on VMD and Improved TCN [J]. Electronic Science and Technology, 2023, 36(3): 42-49.
[11]	ZUO Bin,LI Feifei. An Effective Segmentation Method for COVID-19 CT Image Based on Attention Mechanism and Inf-Net [J]. Electronic Science and Technology, 2023, 36(2): 22-28.
[12]	YU Guangzeng,ZHANG Qiaoling,ZHOU Yurong. Bearing Fault Diagnosis Based on SC-CNN-BiLSTM [J]. Electronic Science and Technology, 2023, 36(11): 56-65.
[13]	ZHANG Wuran,LI Feifei. A 3D Object Detection Network Based on Attention Mechanism and Context Awareness [J]. Electronic Science and Technology, 2023, 36(10): 15-23.
[14]	HUANG Yajing,LIAO Aihua,YU Miao,LI Xiaolong,HU Dingyu. An Improved CNN Method for Bearing Acoustic Fault Diagnosis [J]. Electronic Science and Technology, 2023, 36(1): 75-80.
[15]	DENG Yuan,SHI Yiping,JIANG Yueying,ZHU Yamei,LIU Jin. Infant Expression Recognition Algorithm Based on MobileNetV2 and LBP Feature Fusion [J]. Electronic Science and Technology, 2022, 35(8): 47-52.