基于GAN的无人机航拍图像重建

doi:10.16180/j.cnki.issn1007-7820.2019.08.013

电子科技 ›› 2019, Vol. 32 ›› Issue (8): 61-65.doi: 10.16180/j.cnki.issn1007-7820.2019.08.013

基于GAN的无人机航拍图像重建

曹琨,吴飞,钱小瑞,杨照坤

上海工程技术大学电子电气工程学院,上海 201600

收稿日期:2018-08-13 出版日期:2019-08-15 发布日期:2019-08-12
作者简介:曹琨(1994-),女,硕士研究生。研究方向:深度学习、图像处理。|吴飞(1968-),男,博士,教授。研究方向:计算机组织与系统结构、分布式多媒体技术。
基金资助:
国家自然科学基金(61272097);上海市科技学术委员会资助项目(13510501400);上海市科委重点项目(18511101600)

Aerial Image Reconstruction of Drone Based on GAN

CAO Kun,WU Fei,QIAN Xiaorui,YANG Zhaokun

School of Electronic and Electrical Engineering,Shanghai University of Engineering Science,Shanghai 201600,China

Received:2018-08-13 Online:2019-08-15 Published:2019-08-12
Supported by:
National Natural Science Foundation of China(61272097);Shanghai Municipal Committee of Science and Technology Project(13510501400);Shanghai Municipal Committee of Science and Technology Project(18511101600)

摘要/Abstract

摘要：

传统无人机采集传输过程中所传输的数据量常常造成无人机电池的高消耗。针对此类问题,文中提出一种融合超分辨重建和灰度图像彩色化的CsRGAN模型。通过生成网络对低分辨的灰度图像进行重建:先将图片进行分辨率放大,再进行色彩填充,然后通过判别器进行图片修正,最终将图片重建为彩色高清图像。实验结果表明,在固定区域下,所提出的模型能够在保证成像质量的同时减少无人机航拍的传输数据量,提高无人机的电池利用率,且模型具有较强的鲁棒性。

关键词: 无人机, 超分辨率, 色彩填充, 图像重建, 生成对抗式网络, 固定区域

Abstract:

The amount of data transmitted during the traditional UAV acquisition and transmission process often resulted in high consumption of the UAV battery. Aiming at solving the problem, a CsRGAN model that combined super-resolution reconstruction and gray-scale image colorization was proposed. The low resolution grayscale image was reconstructed by generating a network: the image was first subjected to resolution amplification, color filling was performed, and then the image was corrected by the discriminator, and finally the image was reconstructed into a color high-definition image. The experimental results showed that under the fixed area, the model could reduce the transmission data of the drone aerial photography and improve the battery utilization of the drone under the condition of ensuring the imaging quality. These results proved the model had strong robustness.

Key words: UVA, super-resolution, image colorization, image reconstruction, GAN, fix area

中图分类号:

TP391

曹琨,吴飞,钱小瑞,杨照坤. 基于GAN的无人机航拍图像重建[J]. 电子科技, 2019, 32(8): 61-65.

CAO Kun,WU Fei,QIAN Xiaorui,YANG Zhaokun. Aerial Image Reconstruction of Drone Based on GAN[J]. Electronic Science and Technology, 2019, 32(8): 61-65.

图/表 11

图1

图2

表1

图3

图4

图5

图6

图7

表2

表3

表4

参考文献 16

[1]	管超 . 基于稀疏表示理论的图像超分辨率重构算法研究[D]. 上海:上海交通大学, 2013.
	Guan Chao . Research of image super-resolution reconstruction based on sparse representation theory[D]. Shanghai:Shanghai Jiao Tong University, 2013.
[2]	Han J W, Suryanto, Kim J H , et al. New edge-adaptive image interpolation using anisotropic Gaussian filters[J]. Digital Signal Processing, 2013,23(1):110-117.
[3]	Zhang Q, Wu J . Image super-resolution using windowed ordinary Kriging interpolation[J]. Optics Communications, 2015,33(6):140-145.
[4]	Freeman W T, Jones T R, Pasztor E C . Example-based super-resolution[J]. Computer Graphics & Applications IEEE, 2002,22(2):56-65.
[5]	Zhang K, Gao X, Tao D , et al. Single image super-resolution with non-local means and steering kernel regression[J]. IEEE Transactions on Image Processing, 2012,21(11):4544-56.
[6]	Bevilacqua M, Roumy A, Guillemot C, et al. Neighbor embedding based single-image super-resolution using Semi-Nonnegative Matrix Factorization [C].Kyoto:IEEE International Conference on Acoustics,Speech and Signal Processing, 2012.
[7]	Kim C, Choi K, Ra J B . Example-based super-resolution via structure analysis of patches[J]. IEEE Signal Processing Letters, 2013,20(4):407-410.
[8]	Ledig C, Theis L, Huszar F , et al. Photo-realistic single image super-resolution using a generative adversarial network[C]. Honolulu:30 ^th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2017.
[9]	冈萨雷斯. 数字图像处理[M] 北京: 电子工业出版社, 2005.
	Gonzalez. Digital image processing[M]. Beijing: Publishing House of Electronics Industry, 2005.
[10]	Levin A, Lischinski D, Weiss Y . Colorization using optimization[J]. Image and Vision Computing, 2003,46(3):197-202.
[11]	Yatziv L, Bartesaghi A, Sapiro G . O(N) implementation of the fast marching algorithm[J]. Journal of Computational Physics, 2006,212(2):393-399.
[12]	Welsh T, Ashikhmin M, Mueller K . Transferring color to greyscale images[J]. ACM Transactions on Graphics, 2002,21(3):277-280.
[13]	Isola P, Zhu J Y, Zhou T , et al. Image-to-image translation with conditional adversarial networks[C]. Honolulu:30 ^th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2017.
[14]	Goodfellow I J, Pouget-Abadie J, Mirza M , et al. Generative adversarial networks[J]. Advances in Neural Information Processing Systems, 2014(3):2672-2680.
[15]	Richardson I E G . H.264 and MPEG-4 video compression: video coding for next-generation multimedia[M]. Hoboken: Wiley Press, 2004.
[16]	沈秋, 李小凡, 孔繁锵 , 等. 基于仿射模型的无人机视频实时压缩算法[J]. 电子与信息学报, 2014,36(12):2855-2860. doi: 10.3724/SP.J.1146.2014.00080
	Shen Qiu, Li Xiaofan, Kong Fanqiang , et al. A real-time video compression for UAV based on affine model[J]. Journal of Electronics & Information Technology, 2014,36(12):2855-2860. doi: 10.3724/SP.J.1146.2014.00080

Loss	Per-pixel acc.	Per-class acc.	Class IOU
L₁+GAN	0.68	0.19	0.14
L₁+GAN+L_HSV	0.70	0.22	0.15

序列	输入图像/kB	输出图像/kB	差值倍数
15	27	875	32.4
30	28	935	33.4
81	39	1 356	34.8
130	30	1 057	35.2

序列	保真性PSNR	鲁棒性NC	结构相似性SSIM
15	28.617 190	0.997 791	0.981 915
30	28.815 120	0.997 691	0.981 965
81	27.958 641	0.997 781	0.982 935
130	21.758 541	0.997 715	0.981 915

方法	数据量 /kB·s^-1	传输视频分辨率	播放视频分辨率	300帧视频传输时间/s
文献[15]	10 640.228	640×480	640×480	20.66
文献[16]	12 702.890	640×480	640×480	17.31
本文方法	10 640.228	256×256	1024×1024	2.25

基于GAN的无人机航拍图像重建

Aerial Image Reconstruction of Drone Based on GAN

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 16

相关文章 13

Metrics

本文评价

推荐阅读 10

[1]	张莹,刘子龙,万伟. 基于Faster R-CNN的无人机车辆目标检测[J]. 电子科技, 2021, 34(11): 11-20.
[2]	郭婕,金海,沈昕格. 基于神经网络PID算法的四旋翼无人机优化控制[J]. 电子科技, 2021, 34(10): 51-55.
[3]	杨雨薇,宋芳,章伟. 四旋翼无人机姿态控制传感器故障的区间估计[J]. 电子科技, 2021, 34(10): 56-62.
[4]	张阵委, 章伟, 龙林, 颜晨航. 基于深度相机的小型无人机室内三维地图构建[J]. 电子科技, 2021, 34(1): 65-70.
[5]	黄雨泽,陈广东,王媛,万思钰. 一种共照射源超空间分辨率三维重建方法[J]. 电子科技, 2020, 33(7): 57-62.
[6]	祝光泉,杨海马,李筠,杨萍. 电容层析成像图像重建算法对比[J]. 电子科技, 2020, 33(11): 41-45.
[7]	陈昊,沈景凤,赵程. 无人机在抢险救灾中的优化应用[J]. 电子科技, 2019, 32(9): 20-25.
[8]	郗来迎,赵赫男,王松波,张孝超,马明阳. 基于倾斜摄影技术的无人机电力巡检设计[J]. 电子科技, 2019, 32(5): 85-88.
[9]	王松波,李马骁,李海瑞,孟庆祎,姜明席. 基于三维激光雷达技术的输电线路廊道障碍物检测研究[J]. 电子科技, 2019, 32(4): 81-84.
[10]	岳秀,张伟. 基于智能算法的无人机航迹规划[J]. 电子科技, 2019, 32(2): 9-13.
[11]	夏令儒，孙首群. 多无人机协同任务规划[J]. , 2018, 31(1): 4-.
[12]	沈丽,韩彦芳. 基于稀疏表示的超分辨率图像重建[J]. , 2015, 28(9): 144-.
[13]	粟子芩,徐伯庆,苏栋骐. 图像代数重建算法的并行性研究[J]. , 2015, 28(10): 4-.