融合片内语义和片间结构特征的自监督CT图像分类方法

doi:10.16180/j.cnki.issn1007-7820.2024.07.006

Abstract

Abstract:

In view of the scarcity of artificial labels and poor classification performance in CT(Computed Tomography) image analysis, a self-supervised CT image classification method combining in-slice semantic and interslice structural features is proposed in this study.In this method, the hierarchical structure of CT images and the semantic features of local components are utilized to process the unlabeled lesion images through the confusion section generation algorithm, and the spatial index and confusion section are generated as supervisory information.In the self-supervised auxiliary task, the ResNet50 network was used to extract both the intraslice semantic and interslice structural features related to the lesion site from the confused sections, and the learned features were transferred to the subsequent medical classification task, so that the final model gained from the unlabeled data.The experimental results show that compared with other 2D and 3D models for CT images, the proposed method can achieve better classification performance and label utilization efficiency when the used labeled data is limited.

Key words: medical image classification, 3D medical image processing, CT images, self-supervised learning, transfer learning, few shot learning, intra-slice semantic features, inter-slice structural features, ResNet50

CLC Number:

TP391

CAO Chunping, XU Zhihua. A Self-Supervised CT Image Classification Method Incorporating Intra-Slice Semantic and Inter-Slice Structural Features[J].Electronic Science and Technology, 2024, 37(7): 43-52.

Figures/Tables 11

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Table 1.

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Table 2.

Table 3.

References 38

[1]	闫超, 孙占全, 田恩刚, 等. 基于深度学习的医学图像分割技术研究进展[J]. 电子科技, 2021, 34(2):7-11.
	Yan Chao, Sun Zhanquan, Tian Engang, et al. Research progress of medical image segmentation based on deep learning[J]. Electronic Science and Technology, 2021, 34(2):7-11.
[2]	郑光远, 刘峡壁, 韩光辉. 医学影像计算机辅助检测与诊断系统综述[J]. 软件学报, 2018, 29(5):1471-1514.
	Zheng Guangyuan, Liu Xiabi, Han Guanghui. Survey on medical image computer aided detection and diagnosis systems[J]. Journal of Software, 2018, 29(5):1471-1514.
[3]	陶笃纯. 迈向新世纪的医学影像技术[J]. 中国医学影像技术, 2000, 16(1):1-2.
	Tao Duchun. Medical imaging technology in the new century[J]. Chinese Journal of Medical Imaging Technology, 2000, 16(1):1-2.
[4]	Lecun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[5]	Simonyan K, Zisserman A. Very deep convolutional networks for largescale image recognition[EB/OL].(2015-04-10) [2023-02-15] https://arxiv.org/abs/1409.1556.
[6]	Pan S J, Yang Q. A survey on transfer learning[J]. IE-EE Transactions on Knowledge and Data Engineering, 2009, 22(10):1345-1359.
[7]	Jing L, Tian Y. Selfsupervised visual feature learning with deep neural networks:A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(11):4037-4058.
[8]	赵凯琳, 靳小龙, 王元卓. 小样本学习研究综述[J]. 软件学报, 2021, 32(2):349-369.
	Zhao Kailin, Jin Xiaolong, Wang Yuanzhuo. Survey on few-shot learning[J]. Journal of Software, 2021, 32(2):349-369.
[9]	庄福振, 罗平, 何清, 等. 迁移学习研究进展[J]. 软件学报, 2015, 26(1):26-39.
	Zhuang Fuzhen, Luo Ping, He Qing, et al. Suvery on transfer learning research[J]. Journal of Software, 2015, 26(1):26-39.
[10]	He K, Girshick R, Dollár P. Rethinking imagenet pretraining[C]. Seoul: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019:4918-4927.
[11]	Xu J. A review of self-supervised learning methods in the field of medical image analysis[J]. International Journal of Image,Graphics and Signal Processing, 2021, 13(4):33-46.
[12]	Shurrab S, Duwairi R. Self-supervised learning methods and applications in medical imaging analysis:A survey[J]. PeerJ Computer Science, 2022(8):1045-1052.
[13]	Chen L, Bentley P, Mori K, et al. Self-supervised learning for medical image analysis using image context restoration[J]. Medical Image Analysis, 2019, 58(5):101539-101543.
[14]	Flanders A E, Prevedello L M, Shih G, et al. Construction of a machine learning dataset through collaboration:The RSNA 2019 brain CT hemorrhage challenge[J]. Radiology:Artificial Intelligence, 2020, 2(3):190211-190220.
[15]	Jing L, Tian Y. Self-supervised visual feature learning with deep neural networks:A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(11):4037-4058.
[16]	彭玉姣. 基于自监督学习的图像特征表示方法研究[D]. 合肥: 安徽大学, 2021:7-15.
	Peng Yujiao. Research on image feature representation based on self-supervised learning[D]. Hefei: Anhui University, 2021:7-15.
[17]	Xie Y, Xu Z, Zhang J, et al. Self-supervised learning of graph neural networks:A unified review[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(2):2412-2429.
[18]	Bai W, Chen C, Tarroni G, et al. Self-supervised learning for cardiac MR image segmentation by anatomical position prediction[C]. Shenzhen: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019:541-549.
[19]	Henaff O. Data-efficient image recognition with contrastive predictive coding[C]. Online: International Conference on Machine Learning, 2020:4182-4192.
[20]	Chen T, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C]. Online: International Conference on Machine Learning, 2020:1597-1607.
[21]	He K, Fan H, Wu Y, et al. Momentum contrast for unsupervised visual representation learning[C]. Online: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020:9729-9738.
[22]	Grill J B, Strub F, Altché F, et al. Bootstrap your own latent a new approach to self-supervised learning[J]. Advances in Neural Information Processing Systems, 2020, 33(3):21271-21284.
[23]	Chen X, He K. Exploring simple siamese representation learning[C]. Online: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021:15750-15758.
[24]	Zbontar J, Jing L, Misra I, et al. Barlow twins:Self-supe-rvised learning via redundancy reduction[C]. Online: International Conference on Machine Learning, 2021:12310-12320.
[25]	Ciga O, Xu T, Martel A L. Self supervised contrastive learning for digital histopathology[J]. Machine Learning with Applications, 2022(7):100198-100206.
[26]	Zhou Z, Sodha V, Rahman Siddiquee M M, et al. Models genesis:Generic autodidactic models for 3D medical image analysis[C]. Shenzhen: Medical Image Computing and Computer Assisted Intervention, 2019:384-393.
[27]	Taleb A, Lippert C, Klein T, et al. Multimodal self-supe-rvised learning for medical image analysis[C]. Online: International Conference on Information Processing in Medical Imaging, 2021:661-673.
[28]	Zheng H, Han J, Wang H, et al. Hierarchical self-supervised learning for medical image segmentation based on multi-domain data aggregation[C]. Strasbourg: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2021:622-632.
[29]	Zhuang X, Li Y, Hu Y, et al. Self-supervised feature learning for 3D medical images by playing a rubik's cube[C]. Shenzhen: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019:420-428.
[30]	Noroozi M, Favaro P. Unsupervised learning of visual representations by solving jigsaw puzzles[C]. Amsterdam: Computer Vision-ECCV the Fourteenth European Conference, 2016:69-84.
[31]	Zhu J, Li Y, Hu Y, et al. Embedding task knowledge into 3D neural networks via self-supervised learning[EB/OL].(2020-06-10)[2023-02015]https://arxiv.org/abs/2006.05798.
[32]	Tang Y, Yang D, Li W, et al. Self-supervised pretraining of swin transformers for 3d medical image analysis[C]. New Orleans: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022:20730-20740.
[33]	Ahmed S, Iftekharuddin K M, Vossough A. Efficacy of texture,shape and intensity feature fusion for posterior-fossa tumor segmentation in MRI[J]. IEEE Transactions on Information Technology in Biomedicine, 2011, 15(2):206-213.
[34]	谢凯, 孙鸿飞, 林涛, 等. 影像组学中特征提取研究进展[J]. 中国医学影像技术, 2017, 33(12):1792-1796.
	Xie Kai, Sun Hongfei, Lin Tao, et al. Research progresses in feature extraction of radiomics[J]. Chinese Journal Medical Imaging Technology, 2017, 33(12):1792-1796.
[35]	Yu H, Winkler S. Image complexity and spatial information[C]. Klagenfurt: The Fifth International Workshop on Quality of Multimedia Experience, 2013:12-17.
[36]	郭小英, 李文书, 钱宇华, 等. 可计算图像复杂度评价方法综述[J]. 电子学报, 2020, 48(4):819-826. doi: 10.3969/j.issn.0372-2112.2020.04.024
	Guo Xiaoying, Li Wenshu, Qian Yuhua, et al. Computational evaluation methods of visual complexity perception for images[J]. Acta Electronica Sinica, 2020, 48(4): 819-826. doi: 10.3969/j.issn.0372-2112.2020.04.024
[37]	周兵, 刘玉霞, 杨欣欣, 等. 图像复杂度研究综述[J]. 计算机科学, 2018, 45(9):30-37. doi: 10.11896／j.issn.1002-137X.2018.09.004
	Zhou Bing, Liu Yuxia, Yang Xinxin, et al. Review of research on image complexity[J]. Computer Science, 2018, 45(9):30-37. doi: 10.11896／j.issn.1002-137X.2018.09.004
[38]	Buslaev A, Iglovikov V I, Khvedchenya E, et al. Albumentations:Fast and flexible image augmentations[J]. Information, 2020, 11(2):125-131.

预测类别	真实类别
预测类别	Positive	Negative
Positive	TP(True Positive)	FP(False Positive)
Negative	FN(False Negative)	TN(True Negative)

数据量	Context- Restore-2D	Rubik- Cube-3D	TCPC-3D	本文模型
20%	1.268	1.155	1.308	0.939
40%	0.112	0.739	0.991	0.689
60%	0.938	0.532	0.792	0.467
80%	0.892	0.396	0.524	0.201
100%	0.874	0.328	0.476	0.147
损失降幅	31.07%	71.60%	63.61%	84.35%

数据量	Context- Restore-2D	Rubik- Cube-3D	TCPC-3D	本文模型
20%	0.233	0.167	0.189	0.145
40%	0.191	0.138	0.167	0.109
60%	0.176	0.129	0.142	0.085
80%	0.161	0.110	0.124	0.072
100%	0.153	0.099	0.119	0.065
损失降幅	34.62%	40.7%	37.04%	55.17%

A Self-Supervised CT Image Classification Method Incorporating Intra-Slice Semantic and Inter-Slice Structural Features

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 38

Related Articles 6

Metrics

Comments

Recommended 0

[1]	Bin ,WANG Sen. Visual Detection of Structural Cracks Using Depth Deformable Contour ModelLAI [J]. Electronic Science and Technology, 2023, 36(9): 35-40.
[2]	OU Jingyi,TIAN Ying,XIANG Xin,SONG Qizhe. Fault Diagnosis of Few Shot Industrial Process Based on Transfer BN-CNN Framework [J]. Electronic Science and Technology, 2023, 36(7): 49-55.
[3]	WANG Qiao,HU Chunyan,LI Feifei. Scene Recognition Algorithm Based on Deep Transfer Learning and Multi-Scale Feature Fusion [J]. Electronic Science and Technology, 2023, 36(11): 19-27.
[4]	ZHAI Yuexian,LIU Xiang,SONG Jialin. Staging Diagnosis of Liver Fibrosis Based on Transfer Learning Strategy [J]. Electronic Science and Technology, 2021, 34(6): 11-16.
[5]	XUE Yongjie,JU Zhiyong. Fish Recognition Algorithm Based on Improved AlexNet [J]. Electronic Science and Technology, 2021, 34(4): 12-17.
[6]	CHEN Lei,LI Feifei,CHEN Qiu. Multi-label Image Annotation Algorithm Based on Transfer Learning and Weighted Ranking Support Vector Machine [J]. Electronic Science and Technology, 2020, 33(3): 12-16.