融合上下文感知注意力的低光图像去雾网络

doi:10.19665/j.issn1001-2400.2023.02.003

摘要/Abstract

摘要：

现有的低光去雾算法因受图像光照强度低、光照不均匀等影响,其去雾后的图像存在细节丢失、色彩失真等现象。针对上述问题,提出一种融合上下文感知注意力的低光图像去雾网络(ACANet)。首先,在基准网络中引入层内上下文感知注意力模块,分别从通道维度和空间维度结合全局视角辨识和加权同一尺度下的重要特征,使网络突破局部视野的约束,更加高效地提取图像纹理信息;其次,引入层间上下文感知注意力模块,通过投影操作将高级特征映射到信号子空间,以实现不同层之间多尺度特征信息的高效融合,进一步提升对图像细节的重建;最后,引入CIEDE2000色偏损失函数,通过CIELAB色彩空间对图像色调进行约束,并与L2损失一起联合优化网络,使网络准确地学习图像色彩,以解决图像的严重色偏问题。实验结果表明,所提算法在多个数据集上的客观指标均优于现有去雾算法,其峰值信噪比指标较基准网络提高了8.8%,且主观去雾效果更彻底,恢复图像细节更丰富,色彩还原度更好,更接近于真实图像。

关键词: 低光图像去雾, 注意力机制, 特征融合, 色偏损失, 深度学习

Abstract:

Existing low-light dehazing algorithms are affected by the low and uneven illumination of the hazy images with their dehazed images often suffering from loss of details and color distortion.To address the above problems,a low-light image dehazing network with aggregated context-aware attention (ACANet) is proposed.First,an intra-layer context-aware attention module is introduced to identify and highlight significant features at the same scale from the channel dimension and the spatial dimension,respectively,so that the network can break through the constraints of the local field of view,and extract image texture information more efficiently.Second,an inter-layer context-aware attention module is introduced to efficiently fuse multi-scale features and the advanced features are mapped to the signal subspace through projection operations in order to further enhance the reconstruction of image details.Finally,the CIEDE2000 color shift loss function is adopted to constrain the image hue by CIELAB color space and jointly optimize the network together with L2 loss so as to enable the network to learn image colors accurately and solve the severe color shift problem.Both quantitative and qualitative experimental results on several datasets demonstrate that the proposed ACANet outperforms existing dehazing methods.Specifically,the ACANet improves the PSNR of dehazed images by 8.8% compared to the baseline network,and enhances the image visibility with richer details and more natural color.

Key words: low-light image dehazing, attention mechanism, feature fusion, color shift loss, deep learning

中图分类号:

TP391.41

王柯俨,成吉聪,黄诗芮,蔡坤伦,王威然,李云松. 融合上下文感知注意力的低光图像去雾网络[J]. 西安电子科技大学学报, 2023, 50(2): 23-32.

WANG Keyan,CHENG Jicong,HUANG Shirui,CAI Kunlun,WANG Weiran,LI Yunsong. Low-light image dehazing network with aggregated context-aware attention[J]. Journal of Xidian University, 2023, 50(2): 23-32.

图/表 12

图1

图2

图3

图4

表1

表2

图5

图6

表3

表4

图7

表5

参考文献 20

[1]	孙景荣, 谢林昌, 杜梦欣, 等. 一种非线性变换的自适应透射率去雾算法[J]. 西安电子科技大学学报, 2022, 49(1):208-215.
	SUN Jingrong, XIE Linchang, DU Mengxin, et al. A Nonlinear Transform Adaptive Transmittance Dehazing Algorithm[J]. Journal of Xidian University, 2022, 49(1):208-215.
[2]	LIU Y, WANG A, ZHOU H, JIA P. Single Nighttime Image Dehazing Based on Image Decomposition[J]. Signal Processing, 2021, 183(5):107986. doi: 10.1016/j.sigpro.2021.107986
[3]	JIANG B, MENG H, MA X, et al. Nighttime Image Dehazing with Modified Models of Color Transfer and Guided Image Filter[J]. Multimedia Tools and Applications, 2018, 77(3):3125-3141. doi: 10.1007/s11042-017-4954-9
[4]	ZHANG J, CAO Y, WANG Z. Nighttime Haze Removal Based on a New Imaging Model[C]//2014 IEEE International Conference on Image Processing (ICIP). Piscataway:IEEE, 2014:4557-4561.
[5]	ZHANG J, CAO Y, FANG S, et al. Fast Haze Removal for Nighttime Image Using Maximum Reflectance Prior[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:7418-7426.
[6]	LIU Y, YAN Z, WU A, et al. Nighttime Image Dehazing Based on Variational Decomposition Model[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2022:640-649.
[7]	LIAO Y, SU Z, LIANG X, et al. Hdp-Net:Haze Density Prediction Network for Nighttime Dehazing[C]//Pacific Rim Conference on Multimedia. Heidelberg:Springer, 2018:469-480.
[8]	YAN W, TAN R T, DAI D. Nighttime Defogging Using High-Low Frequency Decomposition and Grayscale-Color Networks[C]//European Conference on Computer Vision. Heidelberg:Springer, 2020:473-488.
[9]	WANG B, HU L, WEI B, et al. Nighttime Image Dehazing Using Color Cast Removal and Dual Path Multi-Scale Fusion Strategy[J]. Frontiers of Computer Science, 2022, 4:147-159.
[10]	DONG H, PAN J, XIANG L, et al. Multi-Scale Boosted Dehazing Network with Dense Feature Fusion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2020:2157-2167.
[11]	ROMANO Y, ELAD M. Boosting of Image Denoising Algorithms[J]. SIAM Journal on Imaging Sciences, 2015, 8(2):1187-1219. doi: 10.1137/140990978
[12]	HARIS M, SHAKHNAROVICH G, UKITA N. Deep Back-Projection Networks for Super-Resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:1664-1673.
[13]	WOO S, PARK J, LEE J Y, et al. CBAM:Convolutional Block Attention Module[C]//Proceedings of the European Conference on Computer Vision(ECCV). Heidelberg:Springer, 2018:3-19.
[14]	刘迪, 郭继昌, 汪昱东, 等. 融合注意力机制的多尺度显著性目标检测网络[J]. 西安电子科技大学学报, 2022, 49(4):118-126.
	LIU Di, GUO Jichang, WANG Yudong, et al. Multi-Scale Saliency Object Detection Network Fused with Attention Mechanism[J]. Journal of Xidian University, 2022, 49(4):118-126.
[15]	CHENG S, WANG Y, HUANG H, et al. Nbnet:Noise Basis Learning for Image Denoising with Subspace Projection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2021:4896-4906.
[16]	HOU Q, ZHOU D, FENG J. Coordinate Attention for Efficient Mobile Network Design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2021:13713-13722.
[17]	SHARMA G, WU W, DALAL E N. The CIEDE2000 Color-Difference Formula:Implementation Notes,Supplementary Test Data,and Mathematical Observations[J]. Color Research & Application,2005, 30(1):21-30.
[18]	ZHANG J, CAO Y, ZHA Z J, et al. Nighttime Dehazing with a Synthetic Benchmark[C]// Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020:2355-2363.
[19]	CHEN D, HE M, FAN Q, et al. Gated Context Aggregation Network for Image Dehazing and Deraining[C]//2019 IEEE Winter Conference on Applications of Computer Vision(WACV). Piscataway:IEEE, 2019:1375-1383.
[20]	QIN X, WANG Z, BAI Y, et al. FFA-Net:Feature Fusion Attention Network for Single Image Dehazing[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2020:11908-11915.

网络参数	数值	参数的含义
batch_size	8	从训练数据中每次抽取的送入网络的图像数量
epochs	400	网络训练的总轮数
lr	0.000 2/0.000 1	学习率
crop_size	256	训练数据裁切之后,送入网络的实际大小
all_T	100	学习率每隔all_T个epoch变为之前的1/10

算法名称	3R数据集		HDP数据集		HDP黄光数据集
算法名称	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
MSBDNet^[10]	24.48	0.950	24.95	0.959	24.94	0.954
GCANet^[19]	20.96	0.901	22.39	0.963	23.94	0.969
FFANet^[20]	24.65	0.949	23.95	0.956	22.56	0.940
Zhang^[5]	13.66	0.770	4.76	0.557	4.82	0.544
osfd^[18]	14.84	0.814	5.74	0.612	5.71	0.589
文中算法	26.64	0.964	26.62	0.962	25.96	0.956

方法名	运行时间/s	运算量	参数量
MSBDNet	0.037	24 536×10⁶	28.7×10⁶
GCANet	0.021	18 397×10⁶	0.7×10⁶
FFANet	0.015	126 706×10⁶	2.0×10⁶
Zhang	0.116
Osfd	0.383
文中网络	0.152	24 732×10⁶	24.7×10⁶

方法名	PSNR/dB	SSIM
(网络B)Baseline	24.48	0.950
(网络C)Baseline+色偏损失	25.66	0.958
(网络D)Baseline+层内上下文感知注意力+色偏损失	26.11	0.960
(网络E)Baseline+层间上下文感知注意力+色偏损失	26.13	0.959
(网络F)文中网络	26.64	0.964

方法名	PSNR/dB	SSIM
Layer-3	24.64	0.953
Layer-4	25.60	0.960
Layer-5	26.64	0.964
Layer-6	26.52	0.963