智能对抗无人机的干扰组合序列生成算法研究

doi:10.19665/j.issn1001-2400.20230903

摘要/Abstract

摘要：

随着无人机自主导航飞行技术的成熟与发展,出现了未经授权的无人机在管制空域随意飞行的现象,给人身安全带来了巨大的隐患,造成了一定程度的经济损失。本研究为在识别无人机飞行状态和实时评估对抗效能的基础上,提高在无人机飞控未知情况下的自适应测控和导航干扰有效性,最终实现基于遥控通信干扰和导航定位干扰多类型干扰组合与非智能无人机的智能对抗博弈。以雷达探测、GPS导航定位、无人机遥控通信压制干扰和GPS导航压制及欺骗干扰等功能构建了反无系统与无人机对抗博弈模型,采用深度强化学习技术和马尔可夫决策过程构建数学模型,同时提出了用于对无人机飞行状态分类的态势评估环为反无系统网络感知干扰效能提供基础信息。分别使用了近端策略优化算法、柔性演员-评论家算法和演员-评论家算法,对构建的智能反无系统进行多次训练,最终生成了依据无人机飞行状态和对抗效能产生智能干扰组合序列的网络参数。所采用的各类深度强化学习算法产生的智能干扰组合序列均实现了欺骗无人机这一最初设定的目标,验证了反无人机系统模型的有效性。对比试验表明,所提态势评估环在反无系统感知干扰效能方面信息是充足有效的。

关键词: 深度强化学习, 无人机, 态势感知, 智能, 对抗, 干扰组合

Abstract:

With the maturity and development of the autonomous navigation flight technology for the unmanned aerial vehicle(UAV),the phenomenon of the unauthorized UAV flying in controlled airspace appears,which brings a great hidden danger to personal safety and causes a certain degree of economic losses.The research of this paper is on improving the effectiveness of adaptive measurement and control and navigation interference in the unknown situation of UAV flight control on the basis of identifying the UAV flight status and real-time evaluation of countermeasure effectiveness,and finally realizing the intelligent countermeasure game between the non-intelligent UAV based on the combination of remote communication interference and navigation and positioning interference.In this paper,a game model of the anti-UAV system(AUS) and UAV confrontation is developed based on the original units of radar detection,GPS navigation positioning,UAV remote communication suppression jamming and GPS navigation suppression and spoofing.The mathematical model is constructed by using deep reinforcement learning and the Markov decision process.Meanwhile,the concept of situation assessment ring for the classification of the UVA flight status is proposed to provide basic information for network sensing jamming effectiveness.The near-end strategy optimization algorithm,maximum entropy optimization algorithm and actor-critic algorithm are respectively used to train the constructed intelligent AUS for many times,and finally the network parameters are generated to generate the intelligent interference combination sequence according to the UAV flight state and countermeasures efficiency.The intelligent interference combination sequences generated by various deep reinforcement learning algorithms in this paper all achieve the initial goal of deceiving UAVs,which verifies the effectiveness of the anti-UAVs system model.The comparison experiment shows that the proposed situation assessment loop is sufficient and effective in the aspect of AUS sensing interference effectiveness.

Key words: deep reinforce learning, UAV, situational awareness, intelligence, confrontation, interference combination

中图分类号:

TN911.7

马小梦, 高梅国, 于默涵, 李云杰. 智能对抗无人机的干扰组合序列生成算法研究[J]. 西安电子科技大学学报, 2023, 50(6): 44-61.

MA Xiaomeng, GAO Meiguo, YU Mohan, LI Yunjie. Research on the interference combinational sequence generation algorithm for the intelligent countermeasure UAV[J]. Journal of Xidian University, 2023, 50(6): 44-61.

图/表 14

图1

图2

图3

图4

图5

表1

系统参数设定"

参数	符号	值
雷达有效探测距离/m	d_e	1 000
距离误差均值/m	μ_d(d_ru)	10
距离误差方差/m²	$σ d 2$	1.0
角度误差参数/(°)	a,b	-0.25、0.25
遥控通信压制干扰距离比阈值	$d c t h$	3/2

表1

表2

表3

图6

图7

图8

图9

图10

图11

参考文献 27

[1]	崔艳鹏, 王元皓, 胡建伟. 一种改进YOLOv3的动态小目标检测方法[J]. 西安电子科技大学学报, 2020, 47(3):1-7.
	CUI Yanpeng, WANG Yuanhao, HU Jianwei. Detection Method for a Dynamic Small Target Using the Improved YOLOv3. Journal of Xidian University, 2020, 47(03):1-7.
[2]	肖尚辉, 陈燕铭, 王梓豫, 等. 协作干扰时频同步误差对安全通信性能影响[J]. 西安电子科技大学学报, 2022, 49(3):1-9.
	XIAO Shanghui, CHEN Yanming, WANG Ziyu, et al. Impact of Cooperative Jamming Time-frequency Synchronization Errors on Secure Communication Performance. Journal of Xidian University, 2022, 49(3):1-9.
[3]	高博, 张乃千, 范旭. 反无人机电子战发展[J]. 国防科技, 2019, 40(1):35-39.
	GAO Bo, ZHANG Naiqian, FAN Xu. Analysis on the Development and Application of Anti-UAV Electronic Warfare[J]. Defense Technology Review, 2019, 40(1):35-39.
[4]	梅丹, 高丽. 定向能武器在反无人机作战中的应用[J]. 中国设备工程, 2017(7):40-42.
	MEI Dan, GAO Li. Application of Directed Energy Weapons in Anti-UAV Warfare[J]. China Plant Engineering, 2017(7):40-42.
[5]	王芳. 自适应跳频通信系统干扰技术研究[D]. 成都: 电子科技大学, 2012.
[6]	李鼎, 尹俊, 谢井. 多频点阻塞式干扰JTIDS系统方法分析[J]. 指挥控制与仿真, 2013, 35(5):121-124.
	LI Ding, YIN Jun, XIE Jin. Analyse of Multi-frequency-point Barrage Jamming on JTIDS[J]. Command Control & Simulation, 2013, 35(5):121-124.
[7]	JIN W C, KIM K, CHOI J W. Robust Jamming Algorithm for Location-based UAV Jamming System[C]// 2019 IEEE Asia-Pacific Microwave Conference(APMC).Piscataway:IEEE, 2019:1581-1583.
[8]	简晨, 尚飞飞. 2.4 GHz频段民用无人机压制干扰研究[J]. 数字通信世界, 2019, 171(3):33-36.
	JIAN Chen, SHANG Feifei. Research on Suppression Interference of Civil UAV in the 2.4 GHz Band[J]. Digital Communication World, 2019, 171(3):33-36.
[9]	刘轶龙, 谭志强, 许磊, 等. 全频段导航压制干扰信号产生技术研究[J]. 现代导航, 2022, 13(1):16-22.
	LIU Yilong, TAN Zhiqiang, XU Lei, et al. Research on Technology of Interference Signal Generation for All Satellite Navigation Frequencies[J]. Modern Navigation, 2022, 13(1):16-22.
[10]	李畅, 王旭东. 基于轨迹欺骗的无人机GPS/INS复合导航系统干扰技术[J]. 南京航空航天大学学报, 2017, 49(3):420-427.
	LI Chang, WANG Xudong. Jamming of Unmanned Aerial Vehicle with GPS/INS Integrated Navigation System Based on Trajectory Cheating[J]. Journal of Nanjing University of Aeronautics and Astronautics, 2017, 49(3):420-427.
[11]	胡洪涛, 李正杰, 兰竹. 基于卫星导航欺骗干扰的无人机诱捕技术[J]. 电子信息对抗技术, 2020, 35(2):48-52.
	HU Hongtao, LI Zhengjie, LAN Zhu. The UAV Trapping Technology Based on Satellite Navigation Spoofing Jamming[J]. Electronic Information Warfare Technology, 2020, 35(2):48-52.
[12]	史鹏亮, 王晓宇, 薛瑞. 无人机位置欺骗诱导策略[J]. 国防科技大学学报, 2021, 43(2):40-46.
	SHI Pengliang, WANG Xiaoyu, XUE Rui. Induction Strategy for Unmanned Aerial Vehicle Position Spoofing[J]. Journal of National University of Defense Technology, 2021, 43(2):40-46.
[13]	郭妍, 唐康华, 张鹭. 面向无人机的逐点偏移式卫星导航欺骗干扰方法[J]. 工程科学与技术, 2023, 55(2):275-284.
	GUO Yan, TANG Kanghua, ZHANG Lu. GNSS Navigation Spoofing Method of UAV Based on Point-by-point Offset[J]. Advanced Engineering Sciences, 2023, 55(2):275-284.
[14]	PARRAS J, ZAZO S, VAL D J, et al. Pursuit-Evasion Games:A Tractable Framework for Antijamming Games in Aerial Attacks[J]. EURASIP Journal on Wireless Communications and Networking, 2017, 69:1-15.
[15]	ELDOSOUKY A, FERDOWSI A; SAAD W. Drones in Distress:A Game-Theoretic Countermeasure for Protecting UAVs Against GPS Spoofing[J]. IEEE Internet of Things Journal, 2020, 7(4):2840-2854. doi: 10.1109/JIoT.6488907
[16]	陈灿, 莫雳, 郑多, 等. 非对称机动能力多无人机智能协同攻防对抗[J]. 航空学报, 2020, 41(12):342-354.
	CHEN Can, MO Pi, ZHENG Duo, et al. Cooperative Attack-defense Game of Multiple UAVs with Asymmetric Maneuverability[J]. Acta Aeronautica ET Astronautica Sinica, 2020, 41(12):342-354.
[17]	李波, 越凯强, 甘志刚, 等. 基于MADDPG的多无人机协同任务决策[J]. 宇航学报, 2021, 42(6):757-765.
	LI Bo, YUE Kaiqiang, GAN Zhigang, et al. Multi-UAV Cooperative Autonomous Navigation Based on Multi-agent Deep Deterministic Policy Gradient[J]. Journal of Astronautics, 2021, 42(6):757-765.
[18]	BAI S X, SONG S M, LIANG S Y, et al. UAV Maneuvering Decision-Making Algorithm Based on Twin Delayed Deep Deterministic Policy Gradient Algorithm[J]. Journal of ArtificialIntelligence and Technology, 2022, 2:16-22.
[19]	郭万春, 解武杰, 尹晖, 等. 基于改进双延迟深度确定性策略梯度法的无人机反追击机动决策[J]. 空军工程大学学报(自然科学版), 2021, 22(4):15-21.
	GUO Wangchun, XIE Wujie, YIN Hui, et al. Research on UAV Anti-pursing Maneuvering Decision Based on Improved Twin Delayed Deep Deterministic Policy Gradient Method[J]. Journal of Air Force Engineering University(Natural Science Edition), 2021, 22(4):15-21.
[20]	符小卫, 徐哲, 朱金冬. 基于PER-MATD3的多无人机攻防对抗机动决策研究(2023)[J/OL].[2023-02-16]. http://kns.cnki.net/kcms/detail/11.1929.V.20220609.1423.002.html.
	FU Xiaowei, XU Zhe, ZHU Jindong, et al. Researchon Maneuvering Decision-making of Multi-UAV Attack-Defence Confrontation Based on PER-MATD3[J/OL].[2023-02-16]. http://kns.cnki.net/kcms/detail/11.1929.V.20220609.1423.002.html.
[21]	王尔申, 刘帆, 宏晨. 基于MASAC的无人机集群对抗博弈方法[J]. 中国科学:信息科学, 2022, 52(12):2254-2269.
	WANG Ershen, LIU Fan, HONG Chen, et al. MASAC-based Confrontation Game Method of UAV Clusters[J]. Science China:Information Sciences, 2022, 52(12):2254-2269.
[22]	YANNIS F B, FERRET J, PIETQUIN O, et al. Adversarially Guided Actor-Critic(2021)[J/OL].[2021-02-08]. https://arxiv.org/abs/2102.04376v1.
[23]	SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal Policy Optimization Algorithms(2017)[J].[2017-07-20]. https://arxiv.org/abs/1707.06347.
[24]	HAARNOJA T, ZHOU A, ABBEEL P, et al. Soft Actor-Critic:Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor(2018)[J/OL].[2018-01-04]. https://arxiv.org/abs/1801.01290v2.
[25]	LI S, CHEN M, WANG Y, et al. Air Combat Decision-Making of Multiple UCAVs Based on Constraint Strategy Games[J]. Defence Technology, 2022, 18(3),368-383. doi: 10.1016/j.dt.2021.01.005
[26]	YANG Y, LI K, LI J, et al. Low-Cost,High-Power Jamming Transmitter Based on Magnetron[J]. IEEE Transactions on Electron Devices, 2020, 67(7),2912-2918. doi: 10.1109/TED.16
[27]	LEI Z, DING P, ZHENG W, et al. UAV Countermeasure Technology Based on Partial-band Noise Jamming[C]// 2021 33rd Chinese Control and Decision Conference(CCDC).Piscataway:IEEE, 2021:1456-1461.

反无系统动作	上一态势	当前态势
反无系统动作	上一态势	靠近		返回		悬停		降落		诱骗
	靠近	-1.0	-0.5	X	1.0	X	X	X	X	X	X
		-1.0	-1.0	-0.5	-0.5	2.0	X	X	X	X	X
0100	返回	1.0	1.0	X	1.0	X	X	X	X	X	X
遥控干扰		-1.0	-1.0	-0.5	-0.5	1.0	X	X	X	X	X
0010	悬停	-0.5	-1.0	X	-1.0	X	X	X	X	X	X
遥控+导航		X	-0.5	X	-1.0	2.0	X	3.0	X	X	X
0001	降落	-2.0	-2.0	X	-1.0	X	X	X	X	X	X
导航诱骗		-1.0	-1.0	-1.0	-1.0	1.0	X	-0.5	X	X	4.0
1000	诱骗	-2.0	-2.0	X	-2.0	-2.0	-1.0	X	X	X	X
无干扰		-2.0	X	-2.0	X	-1.0	4.0	X	X	X	4.0

无人机所受干扰类型	无人机飞行策略
无干扰	进攻目标区域
遥控压制或导航压制	依据概率选择进攻或返航
遥控压制+导航压制	悬停后降落
遥控压制+导航欺骗	飞行至欺骗点