一种聚类辅助的智能频谱分配技术研究

doi:10.19665/j.issn1001-2400.20231006

Abstract

Abstract:

Aiming at the problem of low spectrum utilization of the traditional spectrum allocation scheme in a large-scale and high dynamic electromagnetic spectrum warfare system,intelligent spectrum allocation technology research is carried out.In this paper,first,we construct a complex and highly dynamic electromagnetic spectrum combat scenario,and under the coexistence conditions of multiple types of equipment such as radar,communication and jamming,we model the spectrum allocation of the complex electromagnetic environment as an optimization problem to maximize the number of access devices.Second,an intelligent spectrum allocation algorithm based on clustering assistance is proposed.Aiming at the centralized resource allocation algorithm facing the problem of exploding action space dimensions,a multi-DDQN network is used to characterize the decision-making information of each node.Then based on the elbow law and K-means++ algorithm,a multi-node collaborative approach is proposed,where nodes within a cluster make chained decisions by sharing action information and nodes between clusters make independent decisions,assisting the DDQN algorithm to intelligently allocate resources.By designing the state,action space and reward function,and adopting the variable learning rate to realize the fast convergence of the algorithm,the nodes are able to dynamically allocate the multidimensional resources such as frequency/energy according to the electromagnetic environment changes.Simulation results show that under the same electromagnetic environment,when the number of nodes is 20,the number of accessible devices of the proposed algorithm is increased by about 80% compared with the number by the greedy algorithm,and about 30% compared with that by the genetic algorithm,which is more suitable for the spectrum allocation of multi-devices under dynamic electromagnetic environment.

Key words: dynamic electromagnetic environment, intelligent spectrum allocation, spectrum efficiency, cluster analysis, reinforcement learning

CLC Number:

TN97

ZHAO Haoqin, YANG Zheng, SI Jiangbo, SHI Jia, YAN Shaohu, DUAN Guodong. Research on a clustering-assisted intelligent spectrum allocation technique[J].Journal of Xidian University, 2023, 50(6): 1-12.

Figures/Tables 15

References 20

[1]	王沙飞, 鲍雁飞, 李岩. 认知电子战体系结构与技术[J]. 中国科学:信息科学, 2018, 48(12):1603-1613.
	WANG Shafei, BAO Yanfei, LI Yan. Architecture and Technology of Cognitive Electronic Warfare[J]. Science China:Information Sciences, 2018, 48(12):1603-1613.
[2]	温敬朋, 杨健, 王沙飞. 电子战装备技术发展现状与展望[J]. 信息对抗技术, 2022, 1(1):1-10.
	WEN Jingpeng, YANG Jian, WANG Shafei. Development Status and Prospect of Electronic Warfare Equipment Technology[J]. Information Countermeasure Technology, 2022, 1(1):1-10.
[3]	唐建强, 李昊. 美军电磁战斗管理发展分析[J]. 电子信息对抗技术, 2020, 35(2):39-43.
	TANG Jianqiang, LI Hao. Development Analysis of Electromagnetic Combat Management in US military[J]. Electronic Information Countermeasures Technology, 2020, 35(2):39-43.
[4]	谢爱平, 李复名, 周涛, 等. 基于DoDAF2.0的电磁频谱战体系设计方法[J]. 电子信息对抗技术, 2022, 37(6):72-77.
	XIE Aiping, LI Fuming, ZHOU Tao, et al. Design Method of Electromagnetic Spectrum Warfare System Based on DoDAF2.0[J]. Electronic Information Countermeasures Technology, 2022, 37(6):72-77.
[5]	龙晓波, 张圣鹋, 余晨, 等. 复杂适应性系统-电磁频谱战的解决之道[J]. 中国电子科学研究院学报, 2022, 17(11):1037-1041.
	LONG Xiaobo, ZHANG Shengwu, YU Chen, et al. Complex Adaptive System-Solution of Electromagnetic Spectrum Warfare[J]. Journal of China Academy of Electronic Electronics, 2022, 17(11):1037-1041.
[6]	董晓庆, 程良伦, 郑耿忠, 等. 认知异构无线网络中传输速率最大化的频谱资源分配方法[J]. 通信学报, 2019, 40(9):124-135. doi: 10.11959/j.issn.1000-436x.2019189
	DONG Xiaoqing, CHENG Lianglun, ZHENG Gengzhong, et al. Spectrum Resource Allocation Method for Maximizing Transmission Rate in Cognitive Heterogeneous Wireless Networks[J]. Journal on Communications, 2019, 40(9):124-135. doi: 10.11959/j.issn.1000-436x.2019189
[7]	周杰, 徐梦颖, 王娇娇, 等. 认知无线传感器网络频谱分配的一种改进方法[J]. 西安电子科技大学学报, 2020, 47(3):80-85.
	ZHOU Jie, XU Mengying, WANG Jiaojiao, et al. An Improved Method for Spectrum Allocation in Cognitive Wireless Sensor Networks[J]. Journal of Xidian University, 2020, 47(3):80-85.
[8]	刘辉, 颜飙, 陈永丽. 基于超图的 D2D 多对多资源分配方案[J]. 计算机工程与设计, 2018, 39(12):3605-3609.
	LIU Hui, YAN Biao, CHEN Yongli. D2D Many-to-Many Resource Allocation Scheme Based on Hypergraph[J]. Computer Engineering and Design, 2018, 39(12):3605-3609.
[9]	YANG C C, XU X D, HAN J, et al. GA Based Optimal Resource Allocation and User Matching in Device to Device Underlaying Network[C]// 2014 IEEE Wireless Communications and Networking Conference Workshops(WCNCW).Piscataway:IEEE, 2014:242-247.
[10]	HUANG J, XING C C, QIAN Y, et al. Resource Allocation for Multicell Device-to-Device Communications Underlaying 5G Networks:A Game-Theoretic Mechanism with Incomplete Information[J]. IEEE Transactions on Vehicular Technology, 2017, 67(3):2557-2570. doi: 10.1109/TVT.25
[11]	CHALLITA U, DONG L, SAAD W. Proactive Resource Management for LTE in Unlicensed Spectrum:A Deep Learning Perspective[J]. IEEE Transactions on Wireless Communications, 2018, 17(7):4674-4689. doi: 10.1109/TWC.2018.2829773
[12]	刘建伟, 高峰, 罗雄麟. 基于值函数和策略梯度的深度强化学习综述[J]. 计算机学报, 2019, 42(6):1406-1438.
	LIU Jianwei, GAO Feng, LUO Xionglin. Review of Deep Reinforcement Learning Based on Value Function and Policy Gradient[J]. Chinese Journal of Computers, 2019, 42(6):1406-1438.
[13]	饶宁, 许华, 蒋磊, 等. 基于多智能体深度强化学习的分布式协同干扰功率分配算法[J]. 电子学报, 2022, 50(6):1319. doi: 10.12263/DZXB.20210818
	RAO Ning, XU Hua, JIANG Lei, et al. Distributed Collaborative Interference Power Allocation Algorithm Based on Multi-Agent Deep Reinforcement Learning[J]. Chinese Journal of Electronics, 2022, 50(6):1319. doi: 10.12263/DZXB.20210818
[14]	NAPARSTEK O, COHEN K. Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access[J]. IEEE Transactions on Wireless Communications, 2018, 18(1):310-323. doi: 10.1109/TWC.2018.2879433
[15]	VAN HASSELT H, GUEZ A, SLIVER D. Deep Reinforcement Learning with Double Q-Learning[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2016:2094-2100.
[16]	SI J B, HUANG R, LI Z, et al. When Spectrum Sharing in Cognitive Networks Meets Deep Reinforcement Learning:Architecture,Fundamentals and Challenges[J]. IEEE Network, 2023(Early Access):1-9.
[17]	肖博, 霍凯, 刘永祥. 雷达通信一体化研究现状与发展趋势[J]. 电子与信息学报, 2019, 41(3):739-750.
	XIAO Bo, HUO Kai, LIU Yongxiang. Research Status and Development Trend of Radar Communication Integration[J]. Journal of Electronics & Information Technology, 2019, 41(3):739-750.
[18]	崔新雨, 伍杰, 周一青, 等. 空天地一体化融合组网的挑战与关键技术[J]. 西安电子科技大学学报, 2023, 50(1):1-11.
	CUI Xinyu, WU Jie, ZHOU Yiqing, et al. Challenges and Key Technologies of Air-Space-Ground Integrated Networking[J]. Journal of Xidian University, 2023, 50(1):1-11.
[19]	许华, 宋佰霖, 蒋磊, 等. 一种通信对抗干扰资源分配智能决策算法[J]. 电子与信息学报, 2021, 43(11):3086-3095.
	XU Hua, SONG Bailin, JIANG Lei, et al. An Intelligent Decision Making Algorithm for Communication Counterinterference Resource Allocation[J]. Journal of Electronics & Information Technology, 2021, 43(11):3086-3095.
[20]	BENGIO Y. Practical Recommendations for Gradient-Based Training of Deep Architectures[J]. Neural Networks:Tricks of the Trade:Second Edition, 2012:437-478.

参数	参数值
episode	20 000
每一回合包含时隙数	20
经验池的规模	10 000
mini-batch设置的样本数量X	64
目标Q网络参数更新频率T_fre	200
中心频率/GHz	[0.6,0.9,1.2,1.5,1.8,2.3]
信道带宽/MHz	10
雷达设备功率空间/W	[0,23 000,25 000]
通信设备功率空间/W	[0,10,100]
电磁环境背景噪声功率/W	[10^-12,5×10^-12,10^-11,5×10^-11]
敌方设备干扰功率/W	200
SINR_threshold	1
R_threshold/km	50
战场大小/(km×km)	200×200
栅格密度	20×20
我方节点数量/个	20
敌方节点数量/个	4

干扰等级	干扰强度值/W
等级1	200
等级2	400
等级3	600
等级4	1 000

情况	部分节点坐标
情况1	[12,5] [15,3] [15,7] [17,4]
情况2	[11,3] [13,4] [14,6] [18,5]
情况3	[10,2] [12,4] [13,9] [16,6]
情况4	[13,4] [16,2] [17,5] [18,7]

Research on a clustering-assisted intelligent spectrum allocation technique

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 20

Related Articles 13

Metrics

Comments

Recommended 10

[1]	WANG Xu, SHANG Erke, MIAO Qiguang, DAI Bin, LIU Yang. HCPP:a data efficient hierarchical car-following method [J]. Journal of Xidian University, 2023, 50(6): 161-171.
[2]	WU Yihao,QI Yanli,ZHOU Yiqing,CAI Qing,LIU Ling,SHI Jinglin. UAVs trajectory planning and power allocation based on the convergence of communication,sensing and computing [J]. Journal of Xidian University, 2023, 50(3): 61-74.
[3]	LI Zhongjie,GAO Wei,XIONG Jiyuan,LI Jianghong. DDPG method for joint beamforming and power control in mmwave communication [J]. Journal of Xidian University, 2022, 49(4): 39-48.
[4]	LV Wenkai,YANG Pengfei,DING Yunqing,ZHANG Heyu,ZHENG Tianyang. JEDERL:A task scheduling optimization algorithm for heterogeneous computing platforms [J]. Journal of Xidian University, 2021, 48(6): 67-74.
[5]	LI Teng,CAO Shijie,YIN Siwei,WEI Dawei,MA Xindi,MA Jianfeng. Optimal method for the generation of the attack path based on the Q-learning decision [J]. Journal of Xidian University, 2021, 48(1): 160-167.
[6]	ZHANG Zheming,REN Shuxia,GUO Kaijie. Model of abstractive text summarization for topic-aware communicating agents [J]. Journal of Xidian University, 2020, 47(3): 97-104.
[7]	ZHANG Ying,WEI Minfeng,WANG Shihui,TAO Leiyan,CAO Jian,ZHANG Xing. Aircraft reinforcement learning multi-mode control in orbit [J]. Journal of Xidian University, 2020, 47(2): 75-82.
[8]	ZHOU Yi,CHEN Bo. Method for robot obstacle avoidance based on the improved dueling network [J]. Journal of Xidian University, 2019, 46(1): 46-50.
[9]	YANG Ming;LI Xiang;YANG Hao;LIU Xin;CHEN Kunqi. Joint optimization of cooperative sensing and transmission in energy-efficiency cognitive radio [J]. Journal of Xidian University, 2017, 44(3): 101-107.
[10]	LIU Jiachen;MIAO Qiguang;SONG Jianfeng;CAO Ying . Enhanced one-class learning based on clustering stability analysis [J]. J4, 2015, 42(2): 58-64+121.
[11]	LIU Jie;WANG Yuping. Central force optimization algorithm via clustering simplex search [J]. J4, 2014, 41(6): 100-105.
[12]	ZHANG Wenzhu;SHAO Lina. Dynamic spectrum allocation algorithm for heterogeneous radio networks based on reinforcement learning [J]. J4, 2011, 38(4): 32-37.
[13]	ZHANG Xin1;2;CHANG Yi-lin1;SHEN Zhong1;LIU Fu-jie1;CUI Can1. Fault monitoring policy for a hierarchical network with a multi-manager [J]. J4, 2005, 32(6): 873-876.