融合注意力引导的多尺度低照度图像增强方法

doi:10.19665/j.issn1001-2400.2023.01.015

Abstract

Abstract:

The low light environment affects the image capture equipment,resulting in low contrast,low brightness,and difficulty in distinguishing objects.In order to improve image quality,a method for enhancement of the multi-scale low-light image by combining an attention guidance is proposed.First,a dense residual network is constructed as a multi-scale feature extractor to extract feature maps at different scales in low light images,and the extracted feature maps are fused by using a modified RefineNet,which makes full use of the feature information in the image.Meanwhile,an interpretable attention mechanism is designed to generate an attention graph based on the results of edge detection.Then by combining a loss function the network is guided through training.The purpose is to enhance edge detail information hidden in the dark without increasing the network’s inference burden.Finally,experiments are completed on synthetic images and SID(See-in-the-Dark) datasets,with the results showing that the proposed method can effectively improve brightness and contrast,restore image edge details as well as improve subjective visual effects.Compared to the contrast algorithm,the PSNR and SSIM are improved by at least 0.79dB and 0.119 on average,respectively.

Key words: image processing, image enhancement, attention mechanism, multi-scale feature fusion

CLC Number:

TP391.4

ZHANG Yali, LI Wenyuan, LI Changlu, DING Shaobo. Method for enhancement of the multi-scale low-light image by combining an attention guidance[J].Journal of Xidian University, 2023, 50(1): 129-136.

Figures/Tables 13

References 24

[1]	董如婵, 焦李成, 赵进. 一种深度融合机制的遥感图像目标检测技术[J]. 西安电子科技大学学报, 2021, 48(5):128-138.
	DONG Ruchan, JIAO Licheng, ZHAO Jin. Application of the Deep Fusion Mechanism in Object Detection of Remote Sensing Images[J]. Journal of Xidian University, 2021, 48(5):128-138.
[2]	陈昌川, 王海宁, 黄炼. 一种基于局部表征的面部表情识别算法[J]. 西安电子科技大学学报, 2021, 48(5):100-109.
	CHEN Changchuan, WANG Haining, HUANG Lian. Facial Expression Recognition Based on Local Representation[J]. Journal of Xidian University, 2021, 48(5):100-109.
[3]	曹海杰, 刘宁, 许吉. 红外图像自适应逆直方图增强技术[J]. 红外与激光工程, 2020, 49(4):256-262.
	CAO Haijie, LIU Ning, XU Ji. Adaptive Inverse Histogram Enhancement for Infrared Images[J]. Infrared and Laser Engineering, 2020, 49(4):256-262.
[4]	GUO X J, LI Y, LING H B. LIME:Low-Light Image Enhancement via Illuminate Map Estimation[J]. IEEE Transactions on Image Processing, 2017, 26(2):982-993. doi: 10.1109/TIP.2016.2639450
[5]	蔡秀梅, 马今璐, 吴成茂. 基于模糊同态滤波的彩色图像增强算法[J]. 计算机仿真, 2020, 37(6):342-346.
	CAI Xiumei, MA Jinlu, WU Chengmao. A Color Image Enhancement Algorithm Based on Fuzzy Homomorphic Filtering[J]. Computer Simulation, 2020, 37(6):342-346.
[6]	宋瑞霞, 李达, 余建德. DT-CWT和色调映射相结合的低照度图像增强算法[J]. 计算机辅助设计与图形学学报, 2018, 30(7):1305-1312.
	SONG Ruixia, LI Da, YU Jiande. Low-Illumination Image Enhancement Algorithm Combining DT-CWT and Tone Mapping[J]. Journal of Computer-Aided Design and Graphics, 2018, 30(7):1305-1312.
[7]	LORE K G, AKINTAYO A, SARKAR S. LLNet:A Deep Auto Encoder Approach to Natural Low-Light Image Enhancement[J]. Pattern Recognition, 2015, 61:650-662. doi: 10.1016/j.patcog.2016.06.008
[8]	XIAO Z, ZHANG X, ZHANG F, et al. Diabetic Retinopathy Retinal Image Enhancement Based on Gamma Correction[J]. Journal of Medical Imaging & Health Informatics, 2017(1):149-154.
[9]	WEI C, WANG W, YANG W, et al. Deep Retinex Decomposition for Low-Light Enhancement[EB/OL].[2018-08-14]. https://arxiv.org/abs/1808.04560.
[10]	CHEN C, CHEN Q, XU J, et al. Learning to See in the Dark[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:3291-3300.
[11]	EVAN S, JONATHAN L, TREVOR D, et al. Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651. doi: 10.1109/TPAMI.2016.2572683 pmid: 27244717
[12]	CHEN C, GONG D, WANG H, et al. Learning Spatial Attention for Face Super-Resolution[J]. IEEE Transactions on Image Processing, 2021, 30:1219-1231. doi: 10.1109/TIP.2020.3043093 pmid: 33315560
[13]	ZHANG J, WU J, WANG H, et al. Cloud Detection Method Using CNN Based on Cascaded Feature Attention and Channel Attention[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:1-17.
[14]	HONG S, ZOU Y, WANG W, et al. Weakly Labelled Audio Tagging via Convolutional Networks with Spatial and Channel-Wise Attention[C]//2020 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE, 2020:296-300.
[15]	CANNY J. A Computational Approach to Edge Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8(6):679-698. pmid: 21869365
[16]	LIN G, MILAN A, SHEN C, et al. RefineNet:Multi-Path Refinement Networks for High-Resolution Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(5):1228-1242.
[17]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:770-778.
[18]	DMITRY U, AMDERA V, VICTOR L. Instance Normalization:The Missing Ingredient for Fast Stylization[EB/OL].[2016-07-27]. https://arxiv.org/abs/1607.08022.
[19]	HUANG G, LIU Z, VAN D M L, et al. Densely Connected Convolutional Networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:4700-4708.
[20]	NAIR V, HINTON G. Rectified Linear Units Improve Restricted Boltzmann machines Vinod Nair[C]// International Conference on Machine Learning.Amsterdam:Elsevier, 2010:807-817.
[21]	ARBELAEZ P, MAIRE M, FOWLKES C, et al. Contour Detection and Hierarchical Image Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5):898-916. doi: 10.1109/TPAMI.2010.161 pmid: 20733228
[22]	AGUSTSSON E, TIMOFTE R N. NTIRE 2017 Challenge on Single Image Super-Resolution:Dataset and Study[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:126-135.
[23]	杨观赐, 杨静, 李少波, 等. 基于Dopout与ADAM优化器的改进[J]. 华中科技大学学报(自然科学版), 2018, 46(7):122-127.
	YANG Guanci, YANG Jing, LI Shaobo, et al. Improvement Based on Dopout with ADAM Optimizer[J]. Journal of Huazhong University of Science and Technology(Natural Science Edition), 2018, 46(7):122-127.
[24]	WANG Z, BOVIK A C, SHEIKH H R, et al. Image Quality Assessment:From Error Visibility to Structural Similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4):600-612. doi: 10.1109/TIP.2003.819861

客观指标	LIME	LLNet	SID-Sony	文中
PSNR/dB	24.95	28.04	28.17	29.19
SSIM	0.620	0.718	0.793	0.892

客观指标	LIME	LLNet	SID-Sony	文中
PSNR/dB	25.14	28.09	28.69	29.25
SSIM	0.683	0.720	0.774	0.913

网络	合成图像		SID
网络	PSNR/dB	SSIM	PSNR/dB	SSIM
基础网络	28.940	0.857	29.020	0.886
加入注意力引导后的网络	29.190	0.892	29.250	0.913

Method for enhancement of the multi-scale low-light image by combining an attention guidance

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 24

Related Articles 15

Metrics

Comments

Recommended 10

[1]	LIU Bochong, CAI Huaiyu, YANG Shiyuan, LI Haotian, WANG Yi, CHEN Xiaodong. Lightweight semantic segmentation network for autonomous driving scenarios [J]. Journal of Xidian University, 2023, 50(1): 118-128.
[2]	CUI Shaoguo,CHEN Siqi,DU Xing. Dual graph attention networks model for target sentiment analysis [J]. Journal of Xidian University, 2023, 50(1): 137-148.
[3]	ZHANG Qiang, YANG Xinpeng, ZHAO Shixiang, WEI Dongdong, HAN Zhen. Vehicle-target detection network for SAR images based on the attention mechanism [J]. Journal of Xidian University, 2023, 50(1): 36-47.
[4]	LIU Xiaowen, GUO Jichang, ZHENG Sida. Weakly-supervised salient object detection with the multi-scale progressive network [J]. Journal of Xidian University, 2023, 50(1): 48-57.
[5]	QIN Rui,ZHANG Wei. Multi-scale fire detection algorithm with an anchor free structure [J]. Journal of Xidian University, 2022, 49(6): 111-119.
[6]	LI Jiaojiao, LIU Zhiqiang, SONG Rui, LI Yunsong. Algorithm for segmentation of remote sensing imagery using the improved Unet [J]. Journal of Xidian University, 2022, 49(6): 67-75.
[7]	ZHANG Zhaoyu,TIAN Chunna,ZHOU Heng,TIAN Xilan. Online classification jointed RGBT tracking based on the dual attention Siamese network [J]. Journal of Xidian University, 2022, 49(6): 76-85.
[8]	LI Yangyang, MAO Heting, ZHANG Xiaolong, CHEN Yanqiao, CHAI Xinghua. Small object detection in remote sensing images using non-local context information [J]. Journal of Xidian University, 2022, 49(5): 117-124.
[9]	LIU Shigang,ZHANG Tong,YANG Jiangong,GE Bao. Progressive dialtion residual network for deep binocular stereo matching [J]. Journal of Xidian University, 2022, 49(5): 175-180.
[10]	WANG Kan, WANG Mengyang, LIU Xin, TIAN Guoqiang, LI Chuan, LIU Wei. Event detection by combining self-attention and CNN-BiGRU [J]. Journal of Xidian University, 2022, 49(5): 181-188.
[11]	LIU Di,GUO Jichang,WANG Yudong,ZHANG Yi. Multi-scale salient object detection network combining an attention mechanism [J]. Journal of Xidian University, 2022, 49(4): 118-126.
[12]	GAO Deyong,KANG Zibing,WANG Song,WANG Yangping. Method to recognize human action by using the convolutional block attention mechanism [J]. Journal of Xidian University, 2022, 49(4): 144-155.
[13]	MA Sike,ZHAO Meng,SHI Fan,SUN Xuguo,CHEN Shengyong. Attention driven nuclei segmentation method for cell clusters [J]. Journal of Xidian University, 2022, 49(2): 198-206.
[14]	XU Ying,LIU Shuai,SHAO Meng,YUE Guodong,AN Dong. Multi-scale generation antagonistic network for the low-dose CT images super-resolution reconstruction algorithm [J]. Journal of Xidian University, 2022, 49(2): 228-236.
[15]	SUN Jingrong,XIE Linchang,DU Mengxin,LUO Liyan. Adaptive transmittance dehazing algorithm based on non-linear transformation [J]. Journal of Xidian University, 2022, 49(1): 208-215.