融合注意力引导的多尺度低照度图像增强方法

doi:10.19665/j.issn1001-2400.2023.01.015

摘要/Abstract

摘要：

弱光环境导致图像采集设备拍摄的照片呈现出对比度低、亮度较暗、目标物难以分辨等特点。为了改善图像质量,提出了一种融合注意力引导的多尺度低照度图像增强方法。首先,构建密集残差网络作为多尺度特征提取器,用于提取低照度图像中不同尺度的特征图;其次,利用改进的RefineNet对提取出的不同尺度的特征图进行融合,以便充分利用图像中的特征信息;同时,在网络中引入注意力机制,基于边缘检测结果生成注意力图,并与损失函数相结合来引导网络进行训练,在不增加网络推理负担的同时,增强隐藏在黑暗中的细节信息;最后,实验分别选用合成图像和SID(See-in-the-Dark)数据集进行训练与测试。相较于对比算法,峰值信噪比(PSNR)和结构相似性(SSIM)分别平均提高了约0.79 dB和0.119。结果表明,所提方法能有效提高亮度和对比度,恢复图像边缘细节,主观视觉效果得到提升。

关键词: 图像处理, 图像增强, 注意力机制, 多尺度特征融合

Abstract:

The low light environment affects the image capture equipment,resulting in low contrast,low brightness,and difficulty in distinguishing objects.In order to improve image quality,a method for enhancement of the multi-scale low-light image by combining an attention guidance is proposed.First,a dense residual network is constructed as a multi-scale feature extractor to extract feature maps at different scales in low light images,and the extracted feature maps are fused by using a modified RefineNet,which makes full use of the feature information in the image.Meanwhile,an interpretable attention mechanism is designed to generate an attention graph based on the results of edge detection.Then by combining a loss function the network is guided through training.The purpose is to enhance edge detail information hidden in the dark without increasing the network’s inference burden.Finally,experiments are completed on synthetic images and SID(See-in-the-Dark) datasets,with the results showing that the proposed method can effectively improve brightness and contrast,restore image edge details as well as improve subjective visual effects.Compared to the contrast algorithm,the PSNR and SSIM are improved by at least 0.79dB and 0.119 on average,respectively.

Key words: image processing, image enhancement, attention mechanism, multi-scale feature fusion

中图分类号:

TP391.4

张雅荔, 李文元, 李昌禄, 丁少博. 融合注意力引导的多尺度低照度图像增强方法[J]. 西安电子科技大学学报, 2023, 50(1): 129-136.

ZHANG Yali, LI Wenyuan, LI Changlu, DING Shaobo. Method for enhancement of the multi-scale low-light image by combining an attention guidance[J]. Journal of Xidian University, 2023, 50(1): 129-136.

图/表 13

图1

图2

图3

图4

图5

图6

图7

图8

图9

表1

表2

图10

表3

参考文献 24

[1]	董如婵, 焦李成, 赵进. 一种深度融合机制的遥感图像目标检测技术[J]. 西安电子科技大学学报, 2021, 48(5):128-138.
	DONG Ruchan, JIAO Licheng, ZHAO Jin. Application of the Deep Fusion Mechanism in Object Detection of Remote Sensing Images[J]. Journal of Xidian University, 2021, 48(5):128-138.
[2]	陈昌川, 王海宁, 黄炼. 一种基于局部表征的面部表情识别算法[J]. 西安电子科技大学学报, 2021, 48(5):100-109.
	CHEN Changchuan, WANG Haining, HUANG Lian. Facial Expression Recognition Based on Local Representation[J]. Journal of Xidian University, 2021, 48(5):100-109.
[3]	曹海杰, 刘宁, 许吉. 红外图像自适应逆直方图增强技术[J]. 红外与激光工程, 2020, 49(4):256-262.
	CAO Haijie, LIU Ning, XU Ji. Adaptive Inverse Histogram Enhancement for Infrared Images[J]. Infrared and Laser Engineering, 2020, 49(4):256-262.
[4]	GUO X J, LI Y, LING H B. LIME:Low-Light Image Enhancement via Illuminate Map Estimation[J]. IEEE Transactions on Image Processing, 2017, 26(2):982-993. doi: 10.1109/TIP.2016.2639450
[5]	蔡秀梅, 马今璐, 吴成茂. 基于模糊同态滤波的彩色图像增强算法[J]. 计算机仿真, 2020, 37(6):342-346.
	CAI Xiumei, MA Jinlu, WU Chengmao. A Color Image Enhancement Algorithm Based on Fuzzy Homomorphic Filtering[J]. Computer Simulation, 2020, 37(6):342-346.
[6]	宋瑞霞, 李达, 余建德. DT-CWT和色调映射相结合的低照度图像增强算法[J]. 计算机辅助设计与图形学学报, 2018, 30(7):1305-1312.
	SONG Ruixia, LI Da, YU Jiande. Low-Illumination Image Enhancement Algorithm Combining DT-CWT and Tone Mapping[J]. Journal of Computer-Aided Design and Graphics, 2018, 30(7):1305-1312.
[7]	LORE K G, AKINTAYO A, SARKAR S. LLNet:A Deep Auto Encoder Approach to Natural Low-Light Image Enhancement[J]. Pattern Recognition, 2015, 61:650-662. doi: 10.1016/j.patcog.2016.06.008
[8]	XIAO Z, ZHANG X, ZHANG F, et al. Diabetic Retinopathy Retinal Image Enhancement Based on Gamma Correction[J]. Journal of Medical Imaging & Health Informatics, 2017(1):149-154.
[9]	WEI C, WANG W, YANG W, et al. Deep Retinex Decomposition for Low-Light Enhancement[EB/OL].[2018-08-14]. https://arxiv.org/abs/1808.04560.
[10]	CHEN C, CHEN Q, XU J, et al. Learning to See in the Dark[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:3291-3300.
[11]	EVAN S, JONATHAN L, TREVOR D, et al. Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651. doi: 10.1109/TPAMI.2016.2572683 pmid: 27244717
[12]	CHEN C, GONG D, WANG H, et al. Learning Spatial Attention for Face Super-Resolution[J]. IEEE Transactions on Image Processing, 2021, 30:1219-1231. doi: 10.1109/TIP.2020.3043093 pmid: 33315560
[13]	ZHANG J, WU J, WANG H, et al. Cloud Detection Method Using CNN Based on Cascaded Feature Attention and Channel Attention[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:1-17.
[14]	HONG S, ZOU Y, WANG W, et al. Weakly Labelled Audio Tagging via Convolutional Networks with Spatial and Channel-Wise Attention[C]//2020 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE, 2020:296-300.
[15]	CANNY J. A Computational Approach to Edge Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8(6):679-698. pmid: 21869365
[16]	LIN G, MILAN A, SHEN C, et al. RefineNet:Multi-Path Refinement Networks for High-Resolution Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(5):1228-1242.
[17]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:770-778.
[18]	DMITRY U, AMDERA V, VICTOR L. Instance Normalization:The Missing Ingredient for Fast Stylization[EB/OL].[2016-07-27]. https://arxiv.org/abs/1607.08022.
[19]	HUANG G, LIU Z, VAN D M L, et al. Densely Connected Convolutional Networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:4700-4708.
[20]	NAIR V, HINTON G. Rectified Linear Units Improve Restricted Boltzmann machines Vinod Nair[C]// International Conference on Machine Learning.Amsterdam:Elsevier, 2010:807-817.
[21]	ARBELAEZ P, MAIRE M, FOWLKES C, et al. Contour Detection and Hierarchical Image Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5):898-916. doi: 10.1109/TPAMI.2010.161 pmid: 20733228
[22]	AGUSTSSON E, TIMOFTE R N. NTIRE 2017 Challenge on Single Image Super-Resolution:Dataset and Study[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:126-135.
[23]	杨观赐, 杨静, 李少波, 等. 基于Dopout与ADAM优化器的改进[J]. 华中科技大学学报(自然科学版), 2018, 46(7):122-127.
	YANG Guanci, YANG Jing, LI Shaobo, et al. Improvement Based on Dopout with ADAM Optimizer[J]. Journal of Huazhong University of Science and Technology(Natural Science Edition), 2018, 46(7):122-127.
[24]	WANG Z, BOVIK A C, SHEIKH H R, et al. Image Quality Assessment:From Error Visibility to Structural Similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4):600-612. doi: 10.1109/TIP.2003.819861

客观指标	LIME	LLNet	SID-Sony	文中
PSNR/dB	24.95	28.04	28.17	29.19
SSIM	0.620	0.718	0.793	0.892

客观指标	LIME	LLNet	SID-Sony	文中
PSNR/dB	25.14	28.09	28.69	29.25
SSIM	0.683	0.720	0.774	0.913

网络	合成图像		SID
网络	PSNR/dB	SSIM	PSNR/dB	SSIM
基础网络	28.940	0.857	29.020	0.886
加入注意力引导后的网络	29.190	0.892	29.250	0.913