一种聚类辅助的智能频谱分配技术研究

doi:10.19665/j.issn1001-2400.20231006

摘要/Abstract

摘要：

针对传统频谱分配方案在大规模、高动态电磁频谱战系统中频谱利用率低的问题,开展智能频谱分配技术研究。首先构建复杂高动态电磁频谱作战场景,并在雷达、通信、干扰等多类型设备共存条件下,将复杂电磁环境频谱分配建模为最大化接入设备数量的优化问题。其次,提出一种基于聚类辅助的智能频谱分配算法,针对集中式资源分配算法面临动作空间维度爆炸的问题,使用多DDQN网络表征各节点的决策信息。再基于肘部法则与K-means++算法,提出多节点协同方法,簇内节点通过共享动作信息进行链式决策、簇间节点独立决策,辅助DDQN算法智能分配资源。通过设计状态、动作空间和奖励函数,并采用变学习速率实现算法快速收敛,最终各节点能够根据电磁环境变化,动态分配频/能等多维资源。仿真结果表明:在相同电磁环境下,当节点数为20时,所提算法的可接入设备数较贪婪算法提升了约80%,较遗传算法提升约30%,更适用于动态电磁环境下多设备的频谱分配。

关键词: 动态电磁环境, 智能频谱分配, 频谱效率, 聚类分析, 深度强化学习

Abstract:

Aiming at the problem of low spectrum utilization of the traditional spectrum allocation scheme in a large-scale and high dynamic electromagnetic spectrum warfare system,intelligent spectrum allocation technology research is carried out.In this paper,first,we construct a complex and highly dynamic electromagnetic spectrum combat scenario,and under the coexistence conditions of multiple types of equipment such as radar,communication and jamming,we model the spectrum allocation of the complex electromagnetic environment as an optimization problem to maximize the number of access devices.Second,an intelligent spectrum allocation algorithm based on clustering assistance is proposed.Aiming at the centralized resource allocation algorithm facing the problem of exploding action space dimensions,a multi-DDQN network is used to characterize the decision-making information of each node.Then based on the elbow law and K-means++ algorithm,a multi-node collaborative approach is proposed,where nodes within a cluster make chained decisions by sharing action information and nodes between clusters make independent decisions,assisting the DDQN algorithm to intelligently allocate resources.By designing the state,action space and reward function,and adopting the variable learning rate to realize the fast convergence of the algorithm,the nodes are able to dynamically allocate the multidimensional resources such as frequency/energy according to the electromagnetic environment changes.Simulation results show that under the same electromagnetic environment,when the number of nodes is 20,the number of accessible devices of the proposed algorithm is increased by about 80% compared with the number by the greedy algorithm,and about 30% compared with that by the genetic algorithm,which is more suitable for the spectrum allocation of multi-devices under dynamic electromagnetic environment.

Key words: dynamic electromagnetic environment, intelligent spectrum allocation, spectrum efficiency, cluster analysis, reinforcement learning

中图分类号:

TN97

赵浩钦, 杨政, 司江勃, 石嘉, 严少虎, 段国栋. 一种聚类辅助的智能频谱分配技术研究[J]. 西安电子科技大学学报, 2023, 50(6): 1-12.

ZHAO Haoqin, YANG Zheng, SI Jiangbo, SHI Jia, YAN Shaohu, DUAN Guodong. Research on a clustering-assisted intelligent spectrum allocation technique[J]. Journal of Xidian University, 2023, 50(6): 1-12.

图/表 15

图1

图2

图3

表1

图4

图5

图6

图7

表2

图8

图9

图10

表3

图11

图12

参考文献 20

[1]	王沙飞, 鲍雁飞, 李岩. 认知电子战体系结构与技术[J]. 中国科学:信息科学, 2018, 48(12):1603-1613.
	WANG Shafei, BAO Yanfei, LI Yan. Architecture and Technology of Cognitive Electronic Warfare[J]. Science China:Information Sciences, 2018, 48(12):1603-1613.
[2]	温敬朋, 杨健, 王沙飞. 电子战装备技术发展现状与展望[J]. 信息对抗技术, 2022, 1(1):1-10.
	WEN Jingpeng, YANG Jian, WANG Shafei. Development Status and Prospect of Electronic Warfare Equipment Technology[J]. Information Countermeasure Technology, 2022, 1(1):1-10.
[3]	唐建强, 李昊. 美军电磁战斗管理发展分析[J]. 电子信息对抗技术, 2020, 35(2):39-43.
	TANG Jianqiang, LI Hao. Development Analysis of Electromagnetic Combat Management in US military[J]. Electronic Information Countermeasures Technology, 2020, 35(2):39-43.
[4]	谢爱平, 李复名, 周涛, 等. 基于DoDAF2.0的电磁频谱战体系设计方法[J]. 电子信息对抗技术, 2022, 37(6):72-77.
	XIE Aiping, LI Fuming, ZHOU Tao, et al. Design Method of Electromagnetic Spectrum Warfare System Based on DoDAF2.0[J]. Electronic Information Countermeasures Technology, 2022, 37(6):72-77.
[5]	龙晓波, 张圣鹋, 余晨, 等. 复杂适应性系统-电磁频谱战的解决之道[J]. 中国电子科学研究院学报, 2022, 17(11):1037-1041.
	LONG Xiaobo, ZHANG Shengwu, YU Chen, et al. Complex Adaptive System-Solution of Electromagnetic Spectrum Warfare[J]. Journal of China Academy of Electronic Electronics, 2022, 17(11):1037-1041.
[6]	董晓庆, 程良伦, 郑耿忠, 等. 认知异构无线网络中传输速率最大化的频谱资源分配方法[J]. 通信学报, 2019, 40(9):124-135. doi: 10.11959/j.issn.1000-436x.2019189
	DONG Xiaoqing, CHENG Lianglun, ZHENG Gengzhong, et al. Spectrum Resource Allocation Method for Maximizing Transmission Rate in Cognitive Heterogeneous Wireless Networks[J]. Journal on Communications, 2019, 40(9):124-135. doi: 10.11959/j.issn.1000-436x.2019189
[7]	周杰, 徐梦颖, 王娇娇, 等. 认知无线传感器网络频谱分配的一种改进方法[J]. 西安电子科技大学学报, 2020, 47(3):80-85.
	ZHOU Jie, XU Mengying, WANG Jiaojiao, et al. An Improved Method for Spectrum Allocation in Cognitive Wireless Sensor Networks[J]. Journal of Xidian University, 2020, 47(3):80-85.
[8]	刘辉, 颜飙, 陈永丽. 基于超图的 D2D 多对多资源分配方案[J]. 计算机工程与设计, 2018, 39(12):3605-3609.
	LIU Hui, YAN Biao, CHEN Yongli. D2D Many-to-Many Resource Allocation Scheme Based on Hypergraph[J]. Computer Engineering and Design, 2018, 39(12):3605-3609.
[9]	YANG C C, XU X D, HAN J, et al. GA Based Optimal Resource Allocation and User Matching in Device to Device Underlaying Network[C]// 2014 IEEE Wireless Communications and Networking Conference Workshops(WCNCW).Piscataway:IEEE, 2014:242-247.
[10]	HUANG J, XING C C, QIAN Y, et al. Resource Allocation for Multicell Device-to-Device Communications Underlaying 5G Networks:A Game-Theoretic Mechanism with Incomplete Information[J]. IEEE Transactions on Vehicular Technology, 2017, 67(3):2557-2570. doi: 10.1109/TVT.25
[11]	CHALLITA U, DONG L, SAAD W. Proactive Resource Management for LTE in Unlicensed Spectrum:A Deep Learning Perspective[J]. IEEE Transactions on Wireless Communications, 2018, 17(7):4674-4689. doi: 10.1109/TWC.2018.2829773
[12]	刘建伟, 高峰, 罗雄麟. 基于值函数和策略梯度的深度强化学习综述[J]. 计算机学报, 2019, 42(6):1406-1438.
	LIU Jianwei, GAO Feng, LUO Xionglin. Review of Deep Reinforcement Learning Based on Value Function and Policy Gradient[J]. Chinese Journal of Computers, 2019, 42(6):1406-1438.
[13]	饶宁, 许华, 蒋磊, 等. 基于多智能体深度强化学习的分布式协同干扰功率分配算法[J]. 电子学报, 2022, 50(6):1319. doi: 10.12263/DZXB.20210818
	RAO Ning, XU Hua, JIANG Lei, et al. Distributed Collaborative Interference Power Allocation Algorithm Based on Multi-Agent Deep Reinforcement Learning[J]. Chinese Journal of Electronics, 2022, 50(6):1319. doi: 10.12263/DZXB.20210818
[14]	NAPARSTEK O, COHEN K. Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access[J]. IEEE Transactions on Wireless Communications, 2018, 18(1):310-323. doi: 10.1109/TWC.2018.2879433
[15]	VAN HASSELT H, GUEZ A, SLIVER D. Deep Reinforcement Learning with Double Q-Learning[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2016:2094-2100.
[16]	SI J B, HUANG R, LI Z, et al. When Spectrum Sharing in Cognitive Networks Meets Deep Reinforcement Learning:Architecture,Fundamentals and Challenges[J]. IEEE Network, 2023(Early Access):1-9.
[17]	肖博, 霍凯, 刘永祥. 雷达通信一体化研究现状与发展趋势[J]. 电子与信息学报, 2019, 41(3):739-750.
	XIAO Bo, HUO Kai, LIU Yongxiang. Research Status and Development Trend of Radar Communication Integration[J]. Journal of Electronics & Information Technology, 2019, 41(3):739-750.
[18]	崔新雨, 伍杰, 周一青, 等. 空天地一体化融合组网的挑战与关键技术[J]. 西安电子科技大学学报, 2023, 50(1):1-11.
	CUI Xinyu, WU Jie, ZHOU Yiqing, et al. Challenges and Key Technologies of Air-Space-Ground Integrated Networking[J]. Journal of Xidian University, 2023, 50(1):1-11.
[19]	许华, 宋佰霖, 蒋磊, 等. 一种通信对抗干扰资源分配智能决策算法[J]. 电子与信息学报, 2021, 43(11):3086-3095.
	XU Hua, SONG Bailin, JIANG Lei, et al. An Intelligent Decision Making Algorithm for Communication Counterinterference Resource Allocation[J]. Journal of Electronics & Information Technology, 2021, 43(11):3086-3095.
[20]	BENGIO Y. Practical Recommendations for Gradient-Based Training of Deep Architectures[J]. Neural Networks:Tricks of the Trade:Second Edition, 2012:437-478.

参数	参数值
episode	20 000
每一回合包含时隙数	20
经验池的规模	10 000
mini-batch设置的样本数量X	64
目标Q网络参数更新频率T_fre	200
中心频率/GHz	[0.6,0.9,1.2,1.5,1.8,2.3]
信道带宽/MHz	10
雷达设备功率空间/W	[0,23 000,25 000]
通信设备功率空间/W	[0,10,100]
电磁环境背景噪声功率/W	[10^-12,5×10^-12,10^-11,5×10^-11]
敌方设备干扰功率/W	200
SINR_threshold	1
R_threshold/km	50
战场大小/(km×km)	200×200
栅格密度	20×20
我方节点数量/个	20
敌方节点数量/个	4

干扰等级	干扰强度值/W
等级1	200
等级2	400
等级3	600
等级4	1 000

情况	部分节点坐标
情况1	[12,5] [15,3] [15,7] [17,4]
情况2	[11,3] [13,4] [14,6] [18,5]
情况3	[10,2] [12,4] [13,9] [16,6]
情况4	[13,4] [16,2] [17,5] [18,7]