融合注意力引导的多尺度低照度图像增强方法

doi:10.19665/j.issn1001-2400.2023.01.015

Abstract

Abstract:

The low light environment affects the image capture equipment,resulting in low contrast,low brightness,and difficulty in distinguishing objects.In order to improve image quality,a method for enhancement of the multi-scale low-light image by combining an attention guidance is proposed.First,a dense residual network is constructed as a multi-scale feature extractor to extract feature maps at different scales in low light images,and the extracted feature maps are fused by using a modified RefineNet,which makes full use of the feature information in the image.Meanwhile,an interpretable attention mechanism is designed to generate an attention graph based on the results of edge detection.Then by combining a loss function the network is guided through training.The purpose is to enhance edge detail information hidden in the dark without increasing the network’s inference burden.Finally,experiments are completed on synthetic images and SID(See-in-the-Dark) datasets,with the results showing that the proposed method can effectively improve brightness and contrast,restore image edge details as well as improve subjective visual effects.Compared to the contrast algorithm,the PSNR and SSIM are improved by at least 0.79dB and 0.119 on average,respectively.

Key words: image processing, image enhancement, attention mechanism, multi-scale feature fusion

CLC Number:

TP391.4

ZHANG Yali, LI Wenyuan, LI Changlu, DING Shaobo. Method for enhancement of the multi-scale low-light image by combining an attention guidance[J].Journal of Xidian University, 2023, 50(1): 129-136.

Figures/Tables 13

References 24

[1]	董如婵, 焦李成, 赵进. 一种深度融合机制的遥感图像目标检测技术[J]. 西安电子科技大学学报, 2021, 48(5):128-138.
	DONG Ruchan, JIAO Licheng, ZHAO Jin. Application of the Deep Fusion Mechanism in Object Detection of Remote Sensing Images[J]. Journal of Xidian University, 2021, 48(5):128-138.
[2]	陈昌川, 王海宁, 黄炼. 一种基于局部表征的面部表情识别算法[J]. 西安电子科技大学学报, 2021, 48(5):100-109.
	CHEN Changchuan, WANG Haining, HUANG Lian. Facial Expression Recognition Based on Local Representation[J]. Journal of Xidian University, 2021, 48(5):100-109.
[3]	曹海杰, 刘宁, 许吉. 红外图像自适应逆直方图增强技术[J]. 红外与激光工程, 2020, 49(4):256-262.
	CAO Haijie, LIU Ning, XU Ji. Adaptive Inverse Histogram Enhancement for Infrared Images[J]. Infrared and Laser Engineering, 2020, 49(4):256-262.
[4]	GUO X J, LI Y, LING H B. LIME:Low-Light Image Enhancement via Illuminate Map Estimation[J]. IEEE Transactions on Image Processing, 2017, 26(2):982-993. doi: 10.1109/TIP.2016.2639450
[5]	蔡秀梅, 马今璐, 吴成茂. 基于模糊同态滤波的彩色图像增强算法[J]. 计算机仿真, 2020, 37(6):342-346.
	CAI Xiumei, MA Jinlu, WU Chengmao. A Color Image Enhancement Algorithm Based on Fuzzy Homomorphic Filtering[J]. Computer Simulation, 2020, 37(6):342-346.
[6]	宋瑞霞, 李达, 余建德. DT-CWT和色调映射相结合的低照度图像增强算法[J]. 计算机辅助设计与图形学学报, 2018, 30(7):1305-1312.
	SONG Ruixia, LI Da, YU Jiande. Low-Illumination Image Enhancement Algorithm Combining DT-CWT and Tone Mapping[J]. Journal of Computer-Aided Design and Graphics, 2018, 30(7):1305-1312.
[7]	LORE K G, AKINTAYO A, SARKAR S. LLNet:A Deep Auto Encoder Approach to Natural Low-Light Image Enhancement[J]. Pattern Recognition, 2015, 61:650-662. doi: 10.1016/j.patcog.2016.06.008
[8]	XIAO Z, ZHANG X, ZHANG F, et al. Diabetic Retinopathy Retinal Image Enhancement Based on Gamma Correction[J]. Journal of Medical Imaging & Health Informatics, 2017(1):149-154.
[9]	WEI C, WANG W, YANG W, et al. Deep Retinex Decomposition for Low-Light Enhancement[EB/OL].[2018-08-14]. https://arxiv.org/abs/1808.04560.
[10]	CHEN C, CHEN Q, XU J, et al. Learning to See in the Dark[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2018:3291-3300.
[11]	EVAN S, JONATHAN L, TREVOR D, et al. Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651. doi: 10.1109/TPAMI.2016.2572683 pmid: 27244717
[12]	CHEN C, GONG D, WANG H, et al. Learning Spatial Attention for Face Super-Resolution[J]. IEEE Transactions on Image Processing, 2021, 30:1219-1231. doi: 10.1109/TIP.2020.3043093 pmid: 33315560
[13]	ZHANG J, WU J, WANG H, et al. Cloud Detection Method Using CNN Based on Cascaded Feature Attention and Channel Attention[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:1-17.
[14]	HONG S, ZOU Y, WANG W, et al. Weakly Labelled Audio Tagging via Convolutional Networks with Spatial and Channel-Wise Attention[C]//2020 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE, 2020:296-300.
[15]	CANNY J. A Computational Approach to Edge Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8(6):679-698. pmid: 21869365
[16]	LIN G, MILAN A, SHEN C, et al. RefineNet:Multi-Path Refinement Networks for High-Resolution Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(5):1228-1242.
[17]	HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2016:770-778.
[18]	DMITRY U, AMDERA V, VICTOR L. Instance Normalization:The Missing Ingredient for Fast Stylization[EB/OL].[2016-07-27]. https://arxiv.org/abs/1607.08022.
[19]	HUANG G, LIU Z, VAN D M L, et al. Densely Connected Convolutional Networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:4700-4708.
[20]	NAIR V, HINTON G. Rectified Linear Units Improve Restricted Boltzmann machines Vinod Nair[C]// International Conference on Machine Learning.Amsterdam:Elsevier, 2010:807-817.
[21]	ARBELAEZ P, MAIRE M, FOWLKES C, et al. Contour Detection and Hierarchical Image Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5):898-916. doi: 10.1109/TPAMI.2010.161 pmid: 20733228
[22]	AGUSTSSON E, TIMOFTE R N. NTIRE 2017 Challenge on Single Image Super-Resolution:Dataset and Study[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Piscataway:IEEE, 2017:126-135.
[23]	杨观赐, 杨静, 李少波, 等. 基于Dopout与ADAM优化器的改进[J]. 华中科技大学学报(自然科学版), 2018, 46(7):122-127.
	YANG Guanci, YANG Jing, LI Shaobo, et al. Improvement Based on Dopout with ADAM Optimizer[J]. Journal of Huazhong University of Science and Technology(Natural Science Edition), 2018, 46(7):122-127.
[24]	WANG Z, BOVIK A C, SHEIKH H R, et al. Image Quality Assessment:From Error Visibility to Structural Similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4):600-612. doi: 10.1109/TIP.2003.819861

客观指标	LIME	LLNet	SID-Sony	文中
PSNR/dB	24.95	28.04	28.17	29.19
SSIM	0.620	0.718	0.793	0.892

客观指标	LIME	LLNet	SID-Sony	文中
PSNR/dB	25.14	28.09	28.69	29.25
SSIM	0.683	0.720	0.774	0.913

网络	合成图像		SID
网络	PSNR/dB	SSIM	PSNR/dB	SSIM
基础网络	28.940	0.857	29.020	0.886
加入注意力引导后的网络	29.190	0.892	29.250	0.913

Method for enhancement of the multi-scale low-light image by combining an attention guidance

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 24

Related Articles 15

Metrics

Comments

Recommended 0

[1]	QU Jiahui, HE Jie, DONG Wenqian, LI Yunsong, ZHANG Tongzhen, YANG Yufei. Change detection method based on multi-scale and multi-resolution information fusion [J]. Journal of Xidian University, 2025, 52(1): 105-116.
[2]	LI Linke, CHEN Jie, LIU Jun. Improved schemes and applications of the neural network differential distinguisher [J]. Journal of Xidian University, 2025, 52(1): 196-214.
[3]	TANG Shuyuan, ZHOU Yiqing, LI Jintao, LIU Chang, SHI Jinglin. Dual attention pedestrian detector for occlusion scenario based on feature calibration [J]. Journal of Xidian University, 2024, 51(6): 25-39.
[4]	SUN Feifan, ZHAO Yan. Image hashing combining adaptive grid descriptor and image energy [J]. Journal of Xidian University, 2024, 51(5): 136-148.
[5]	WANG Xiaopeng, SHI Huan. Fall detection algorithm based on the improved YOLOv8 combined with key points [J]. Journal of Xidian University, 2024, 51(5): 149-164.
[6]	CHEN Kejia, ZHANG Yupeng, LIN Hongxi. Aspect-based sentiment analysis of syntactic perception and knowledge enhancement [J]. Journal of Xidian University, 2024, 51(5): 165-178.
[7]	LIANG Liming, DONG Xin, LEI Kun, XIA Yuchen, WU Jian. Retinal image quality grading for fused attention spectrum non-local blocks [J]. Journal of Xidian University, 2024, 51(4): 102-113.
[8]	LI Xiaohan, YANG Yanbo, ZHANG Jiawei, LI Baoshan, MA Jianfeng. Graph neural network vulnerability detection for ethernet smart contracts [J]. Journal of Xidian University, 2024, 51(4): 139-150.
[9]	GUAN Yepeng, SU Guangyao, SHENG Yi. Time series prediction method based on the bidirectional long short-term memory network [J]. Journal of Xidian University, 2024, 51(3): 103-112.
[10]	XIA Yilan, WANG Xiumei, CHENG Peitao. Texture-aware video inpainting algorithm based on the multi-attention mechanism [J]. Journal of Xidian University, 2024, 51(3): 136-146.
[11]	DING Xinmiao, WANG Jiaxing, GUO Wen. Three-dimensional attention-enhanced algorithm for violence scene detection [J]. Journal of Xidian University, 2024, 51(1): 114-124.
[12]	ZHONG Hao, BIAN Shan, WANG Chuntao. Real world image tampering localization combining the self-attention mechanism and convolutional neural networks [J]. Journal of Xidian University, 2024, 51(1): 135-146.
[13]	ZHENG Kengtao, LI Bin, ZENG Jinhua. Document image forgery localization and desensitization localization using the attention mechanism [J]. Journal of Xidian University, 2023, 50(6): 207-218.
[14]	XIONG Jingwei, PAN Jifei, BI Daping, DU Mingyang. Multi-scale convolutional attention network for radar behavior recognition [J]. Journal of Xidian University, 2023, 50(6): 62-74.
[15]	YANG Yanyan,DU Yanhui,LIU Hongmeng,ZHAO Jiapeng,SHI Jinqiao,WANG Xuebin. Dark web author alignment based on attention augmented convolutional networks [J]. Journal of Xidian University, 2023, 50(4): 206-214.