基于Swin-Transformer的颈动脉超声图像斑块分割

doi:10.16180/j.cnki.issn1007-7820.2024.09.008

摘要/Abstract

摘要：

评估颈动脉超声图像斑块需要大量且经验丰富的临床医生,并且超声图像具有边界模糊、噪声干扰强等特性,使得评估斑块耗时费力,因此需要一种全自动的颈动脉斑块分割方法来解决人力稀缺问题。文中提出了一种基于Swin-Transformer(Shifted-Windows Transformer)模块的深度神经网络模型用于自动分割颈动脉斑块。在U-Net(U-Convolutional Network)架构基础上,编码部分使用3个用于图像下采样的卷积块以获得不同分辨率大小的特征图像,再添加6对连续Swin-Transformer模块用于更细化的特征提取。解码部分将Swin-Transformer模块输出的细化特征逐级上采样,分别与编码部分各级分辨率的特征图进行跳跃连接。基于同仁医院数据集进行对比实验,结果显示所提网络模型Dice指标达到0.814 2,高于其他对比网络,证明了所提模型可以有效地提取颈动脉超声图像斑块特征,实现自动化、高精度的斑块分割。

关键词: 颈动脉斑块, 深度学习, 超声图像, 图像处理, 分割算法, 医学图像, U型架构, Swin-Transformer模块

Abstract:

The evaluation of carotid ultrasound image plaque requires a large number of experienced clinicians, and the ultrasound image has the characteristics of blurred boundary and strong noise interference, making the evaluation of plaques time-consuming and laborious. Therefore, a fully automated carotid plaque segmentation method is urgently needed to solve the problem of manpower scarcity. This study proposes a deep neural network model based on Swin-Transformer (Shifted-Windows Transformer) block for the automatic segmentation of carotid plaques. Based on the U-Net(U-Convolutional Network) architecture, the encoding part uses three convolutional blocks for image down-sampling to obtain feature images of different resolution sizes, and then adds six pairs of two consecutive Swin-Transformer blocks for more refined feature extraction. The decoding part up-samples the refined features output by the Swin-Transformer module step by step, and jump-joints them with the feature maps of each resolution level in the encoding part, respectively. The comparison experiments based on the data set of Tong Ren Hospital show that the Dice index of the proposed deep neural network model reaches 0.814 2, which is higher than that of other comparison networks. The results demonstrate that the proposed model can effectively extract the features of carotid ultrasound image plaques and achieve automated and high-precision plaque segmentation.

Key words: carotid plaque, deep learning, ultrasound image, image processing, segmentation algorithm, medical image, U-shaped architecture, Swin-Transformer block

中图分类号:

TP391.41

何志强, 孙占全. 基于Swin-Transformer的颈动脉超声图像斑块分割[J]. 电子科技, 2024, 37(9): 48-56.

HE Zhiqiang, SUN Zhanquan. Swin-Transformer-Based Carotid Ultrasound Image Plaque Segmentation[J]. Electronic Science and Technology, 2024, 37(9): 48-56.

图/表 10

图1

图2

图3

图4

图5

图6

表1

图7

表2

图8

参考文献 26

[1]	Al-Mamari A. Atherosclerosis and physical activity[J]. Oman Medical Journal, 2009, 24(3):173-181. doi: 10.5001/omj.2009.34 pmid: 22224180
[2]	Barnett P A, Spence J D, Manuck S B, et al. Psychological stress and the progression of carotid artery disease[J]. Journal of Hypertension, 1997, 15(1):49-55. pmid: 9050970
[3]	Akkus Z, Carvalho D, Oord S, et al. Fully automated carotid plaque segmentation in combined cont-rast-enhanced and Bmode ultrasound[J]. Ultrasound in Medicine and Biology, 2015, 41(2):517-531.
[4]	Benjamin E J, Virani S S, Callaway C W, et al. Heart disease and stroke statistics-2019 update:A report from the american heart association[J]. Cirulation, 2019, 139(10):506-528.
[5]	He K, Lian C, Zhang B, et al. HF-UNet:Learning hierarchically inter-task relevance in multi-task U-Net for accurate prostate segmentation in CT images[J]. IEEE Transaction on Medical Imaging, 2021, 40(8):2118-2128.
[6]	Zhou R, Guo F, Azarpazhooh M, et al. A voxel-based fully convolution network and continuous max-flow for carotid vessel-wall-volume segmentateon from 3D ultrasound images[J]. IEEE Transaction on Medical Imaging, 2020, 39(9):2844-2855.
[7]	Hossain M M, AlMuhanna K, Zhao L, et al. Semia-utomatic segmentation of atherosclerotic carotid artery wall volume using 3D ultrasound imaging[J]. Medical Physics, 2015, 42(4):2029-2037. doi: 10.1118/1.4915925 pmid: 25832093
[8]	司明明, 陈玮, 胡春燕, 等. 融合Resnet50和U-Net的眼底彩色血管图像分割[J]. 电子科技, 2021, 34(8):19-24.
	Si Mingming, Chen Wei, Hu Chunyan, et al. Fundus blood vessel image segmentation combining Resnet50 and U-Net[J]. Electronic Science and Technology, 2021, 34(8):19-24.
[9]	Liguori C, Paolillo A, Pirtrosanto A. An automatic measurement system for the evaluation of carotidintima-media thickness[J]. IEEE Transactions on Instrumentation and Measurement, 2001, 50(6):1684-1691.
[10]	Destrempes F, Meunier J, Giroux M, et al. Segmentation of plaques in sequences of ultrasonic B mode images of carotid arteries based on motion estimation and a Bayesian model[J]. IEEE Transactions on Biomedical Engineering, 2011, 58(8):2202-2211.
[11]	Delsanto S, Molinari F, Giustetto P, et al. Characterization of a completely user-independent algorithm for carotid artery segmentation in 2D ultrasound images[J]. IEEE Transactions on Instrumentation and Measurement, 2007, 56(4):1265-1274.
[12]	Bonanno L, Sottile F, Ciurleo R, et al. Automatic algorithm for segmentation of atheroscle-rotic carotid plaque[J]. Journal of Stroke and Cerebrovascular Diseases, 2017, 26(2):411-416.
[13]	Menchon-Lara R M, Sancho-Gomez J L, Bueno-C-respo A. Early-stage atherosclerosis detection using deep learning over carotid ultrasound images[J]. Applied Soft Computing, 2016, 49(1):616-628.
[14]	Shen D, Wu G, Suk H I. Deep learning in medical image analysis[J]. Annual Review of Biomedical Engineering, 2017, 19(7):221-248.
[15]	Zhou R, Fenster A, Xia Y, et al. Deep learning-based carotid media-adventitia and lumen-intima boundary segmentation from three-dimensional ultrasound images[J]. Medical Physics, 2019, 46(7):3180-3193. doi: 10.1002/mp.13581 pmid: 31071228
[16]	Azzopardi C, Camilleri K P, Hicks Y A. Bimodal automated carotid ultrasound segmentation using geometrically constrained deep neural networks[J]. IEEE Journal of Biomedical Health Informatic, 2020, 24(4):1004-1015.
[17]	Mi S, Bao Q, Wei Z, et al. MBFF-Net:Multi-branch feature fusion network for carotid plaque segmentation in ultrasound[C]. Strasbourg: Medical Image Computing and Computer Assisted Intervention the Twenty-fourth International Conference, 2021:313-322.
[18]	闫超, 孙占全, 田恩刚, 等. 基于深度学习的医学图像分割技术研究进展[J]. 电子科技, 2021, 34(2):7-11.
	Yan Chao, Sun Zhanquan, Tian Engang, et al. Research progress of medical image segmentation based on deep learning[J]. Electronic Science and Technology, 2021, 34(2):7-11.
[19]	Ronneberger O, Fischer P, Brox T. U-net:Convolutional networks for biomedical image segmentation[C]. Munich: Medical Image Computing and Computer-Assisted Intervention the Eighteenth International Conference, 2015:234-241.
[20]	Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network[C]. Holunono: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:2881-2890.
[21]	Gu C, Cheng G, Fu H, et al. CE-Net:Context encoder network for 2D medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2019, 38(10):2281-2292.
[22]	Yang M, Yu K, Zhang C, et al. Denseaspp for semantic segmentation in street scenes[C]. Salt Lack City: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018:3684-3692.
[23]	Yurtkulu S C, Sahin Y H, Unal G. Semantic segmentation with extended DeepLabv3 architecture[C]. Sivas: IEEE the Twenty-seventh Signal Processing and Communications Applications Conference, 2019:1-4.
[24]	Lin G, Milan A, Shen C, et al. RefineNet:Multi-path refinement networks for high-resolution semantic segmentation[C]. Holunono: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017:1925-1934.
[25]	Liu Z, Lin Y, Cao Y, et al. Swin transformer:Hierarchical vision transformer using shifted windows[C]. Montreal: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021:10012-10022.
[26]	Abraham N, Khan N M. A novel focal tversky loss function withimproved attention U-Net for lesion segmentation[C]. Venice: IEEE the Sixteenth International Symposium on Biomedical Imaging, 2019:683-687.

方法	Dice	Precision	Recall	IoU
Baseline	0.689 7	0.744 1	0.657 9	0.563 1
Baseline+12 Transformer	0.793 8	0.842 7	0.756 3	0.682 4
Baseline+6 S-Transformer	0.806 7	0.849 4	0.791 4	0.689 1
本文	0.814 2	0.852 2	0.796 4	0.703 0

方法	Dice	Recall	IoU
UNet	0.689 7	0.657 9	0.563 1
RefineNet	0.714 3	0.695 2	0.614 2
AttUnet	0.715 9	0.750 1	0.605 6
CeNet	0.742 1	0.689 2	0.635 8
DeeplabV3+	0.771 8	0.785 5	0.656 6
PspNet	0.779 5	0.768 5	0.662 1
DenseAspp	0.784 1	0.775 6	0.679 8
DeeplabV3	0.795 6	0.787 7	0.693 4
本文	0.814 2	0.796 4	0.703 0