一种多尺度GAN的低剂量CT超分辨率重建方法

doi:10.19665/j.issn1001-2400.2022.02.026

摘要/Abstract

摘要：

低剂量CT可以降低X射线辐射、减少对人体的伤害,但成像质量也会显著下降。为得到具有精细结构细节的高质量成像,提出一种基于多尺度残差生成网络的低剂量CT图像超分辨率重建算法,在保持病理不变情况下,从低分辨率图像中恢复高分辨率图像。首先,多尺度网络可以充分利用不同大小图像特征,丰富图像细节信息,提高重建过程中对特征的利用率;其次,引入残差网络,在实现特征重复利用的同时能够很好地防止过拟合现象;最后,将对抗损失和内容损失相结合,约束特征生成的同时能够获得感知质量更好的重建图像。结果表明,该方法在结构相似性、特征相似性、峰值信噪比指标上分别约提高0.047、0.022 8和1.962,算法的IS、FID、SWD性能也比其他两种基于生成对抗网络算法要好,并且在边缘轮廓细节面有更好的表现。为了验证内容损失的有效性,将MSRGAN和其3种变化模型进行比较:MSRGAN相对其变化模型在结构相似性指标平均高出4%,特征相似性指标平均高出3.1%,峰值信噪比指标平均高出17.4%,说明了这种损失函数能够提高超分辨率图像的感知质量,充分证明所提算法的有效性。

关键词: 图像处理, 超分辨率, 生成对抗网络, 多尺度特征, 低剂量CT

Abstract:

Low-dose CT can reduce X-ray radiation and damage to human body,but the imaging quality can also be significantly reduced.In order to obtain high quality images with fine structural details,a low dose CT image super-resolution reconstruction algorithm based on the multi-scale residual generation network (MSRGAN) is proposed to recover high resolution (HR) images from low resolution (LR) images with pathology unchanged.First,a residual network is introduced to prevent overfitting while realizing feature reuse.Second,the multi-scale network can make full use of image features of different sizes,enrich image details and improve the utilization rate of features in the reconstruction process.Finally,by combining the adversarial loss and content loss,the reconstructed image with a better perceived quality can be obtained when the generated feature is constrained.Experimental results show that compared with other algorithms,this method improves in SSIM,FSIM and PSNR indexes,that its GAN's IS,FID and SWD performance is better than that of the other two Gan-based algorithms,and that it has a better performance in edge contour detail,which fully proves the effectiveness of this algorithm.

Key words: image processing, super-resolution, generate antagonistic network, multi-scale features, low-dose CT

中图分类号:

TP391

须颖,刘帅,邵萌,岳国栋,安冬. 一种多尺度GAN的低剂量CT超分辨率重建方法[J]. 西安电子科技大学学报, 2022, 49(2): 228-236.

XU Ying,LIU Shuai,SHAO Meng,YUE Guodong,AN Dong. Multi-scale generation antagonistic network for the low-dose CT images super-resolution reconstruction algorithm[J]. Journal of Xidian University, 2022, 49(2): 228-236.

图/表 10

图1

图2

图3

图4

图5

图6

表1

表2

表3

表4

参考文献 21

[1]	DONG C, LOY C C, HE K, et al. Image Super-Resolution Using Deep Convolutional Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307. doi: 10.1109/TPAMI.2015.2439281
[2]	SHI W Z, CABALLERO J, HUSZÁR F, et al. Real-Time Single Image and Video Super Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway:IEEE, 2016:1874-1883.
[3]	KIM J, LEE J K, LEE K M. Accurate Image Super-Resolution Using Very Deep Convolutional Networks[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway:IEEE, 2016:1646-1654.
[4]	MAO X, SHEN C, YANG Y, et al. Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections[C]// Proceedings of International Conference on Neural Information Processing Systems.Heidelberg:Springer, 2016:2810-2818.
[5]	LI J C, FANG F, MEI K F, et al. Multi-Scale Residual Network for Image Super-Resolution[C]// Proceedings of 2018 ECCV European Conference on Computer Vision.Heidelberg:Springer, 2018:527-542.
[6]	PARK S J, SON H, CHO S H. et al. SRFeat:Single Image Super-Resolution with Feature Discrimination[C]// Proceedings of 2018 ECCV European Conference on Computer Vision.Heidelberg:Springer, 2018:455-471.
[7]	BULAT A, YANG J, GEORGIOS T. To Learn Image Super-Resolution,Use a GAN to Learn How to do Image Degradation First[C]// Proceedings of 2018 ECCV European Conference on Computer Vision.Heidelberg:Springer, 2018:187-202.
[8]	LEDIG C, THEIS L, HUSZARF, et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway:IEEE, 2017:105-114.
[9]	FRID-ADAR M, DIAMANT I, KLANG E, et al. GAN-Based Synthetic Medical Image Augmentation for Increased CNN Performance in Liver Lesion Classification[J]. Neurocomputing, 2018, 321:321-331. doi: 10.1016/j.neucom.2018.09.013
[10]	ISOLA P, ZHU J, ZHOU T, et al. Image-To-Image Translation with Conditional Adversarial Networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Piscataway:IEEE, 2017:5967-5976.
[11]	WU Z, SHEN C, DEN HENGELA V. Wider or Deeper:Revisiting the ResNet Model for Visual Recognition[J]. Pattern Recognition, 2019, 90:119-133. doi: 10.1016/j.patcog.2019.01.006
[12]	ODENA A, DUMOULIN V, OLAH C, et al. Deconvolution and Checkerboard Artifacts(2016)[EB/OL].[2016-10-17]https://distill.pub/2016/deconv-checkerboard/?utm_source=researcher_app&utm_medium=referral&utm_campaign=RESR_MRKT_Researcher_inbound.
[13]	SHI W, CABALLERO J, HUSZáR F, et al. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Piscataway:IEEE, 2016:1874-1883.
[14]	WANG Z, JIANG K, YI P, et al. Ultra-Dense GAN for Satellite Imagery Super-Resolution[J]. Neurocomputing, 2020, 398:328-337. doi: 10.1016/j.neucom.2019.03.106
[15]	张杨忆, 林泓, 管钰华, 等. 改进残差块和对抗损失的GAN图像超分辨率重建[J]. 哈尔滨工业大学学报, 2019, 51(11):128-137.
	ZHANG Yangyi, LIN Hong, GUAN Yuhua, et al. GAN Image Super-Resolution Reconstruction Model with Improved Residual Block and Adversarial Loss[J]. Journal of Harbin Institute of Technology, 2019, 51(11):128-137.
[16]	ZHOU D, DUAN R, ZHAO L, et al. Single Image Super-Resolution Reconstruction Based on Multi-Scale Feature Mapping Adversarial Network[J]. Signal Processing, 2020, 166:107251. doi: 10.1016/j.sigpro.2019.107251
[17]	KASEM H M, HUNG K W, JIANG J. Spatial Transformer Generative Adversarial Network for Robust Image Super-Resolution[J]. IEEE Access, 2019, 7:182993-183009. doi: 10.1109/ACCESS.2019.2959940
[18]	MAHAPATRA D, BOZORGTABAR B, GARNAVI R, et al. Image Super-Resolution Using Progressive Generative Adversarial Networks for Medical Image Analysis[J]. Computerized Medical Imaging and Graphics, 2019, 71:30-39. doi: 10.1016/j.compmedimag.2018.10.005
[19]	ZHANG K, TAO D,GAOX, et al. Coarse-to-Fine Learning for Single-Image Super-Resolution[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(5):1109-1122. doi: 10.1109/TNNLS.2015.2511069
[20]	DONG C, LOY CC, TANG X, et al. Accelerating the Super-Resolution Convolutional Neural Network[C]// European Conference on Computer Vision.Heidelberg:Springer, 2016:391-407.
[21]	YOU C, LI G, ZHANG Y, et al. CT Super-Resolution GAN Constrained by the Identical,Residual,and Cycle Learning Ensemble (GAN-CIRCLE)[J]. IEEE Transactions on Medical Imaging, 2020, 39(1):188-203. doi: 10.1109/TMI.2019.2922960

重建图像	A	B	C	D	E	F	G	H
CTF	3.82	3.90	3.81	3.97	3.87	3.91	3.93	3.85
FSRCNN	4.03	3.94	3.96	4.13	3.99	4.05	4.11	4.09
RDN	3.98	4.01	4.12	4.24	4.12	4.16	4.21	4.19
SRGAN	4.26	4.16	4.22	4.31	4.23	4.32	4.28	4.17
GAN-CIRCLE	4.37	4.35	4.41	4.39	4.43	4.38	4.41	4.36
MSRGAN	4.42	4.37	4.46	4.48	4.45	4.39	4.51	4.43

重建图像	SSIM	FSIM	PSNR	耗时控制/s
CTF	0.892	0.945	28.32	0.008 1
FSRCNN	0.903	0.952	29.15	1.213 6
RDN	0.843	0.960	30.48	1.312 5
SRGAN	0.836	0.971	29.86	1.423 7
GAN-CIRCLE	0.911	0.968	31.73	1.416 3
MSRGAN	0.924	0.982	31.87	1.409 1

模型	IS	FID	SWD(×103)
Real images	8.72	1.43	2.57
SRGAN	4.87	13.62	7.91
GAN-CIRCLE	5.93	7.04	6.86
MSRGAN	6.14	6.85	6.81

模型	SSIM	FSIM	PSNR
MSRGAN	0.924	0.982	31.87
MSRGAN-m	0.882	0.958	30.26
MSRGAN-v	0.876	0.943	29.98
MSRGAN-d	0.894	0.952	30.15