基于深度强化学习的异构融合网络接入算法

doi:10.16180/j.cnki.issn1007-7820.2024.11.002

Abstract

Abstract:

With the increasingly mature communication networks in the fields of air, space and ground, cross-domain heterogeneous converged technology has become an important direction for the integrated development of future communication networks. Driven by the demand for cross-domain heterogeneous in the converged network of air, space and ground, this study aims to solve the problem of low spectrum resource utilization in heterogeneous networks. It uses deep reinforcement learning method to establish a heterogeneous converged network system model and designs intelligent agent access algorithm with fair scale. The system throughput is selected as the maximization objective. The communication network standards that meet the characteristics of air, space and ground integration are selected and corresponding access protocols are extracted. Non-dimensional channel parameters are set according to the principle of fairness and simulation scenarios are established. Multiple comparison strategies are introduced in the simulation to statistically analyze system throughput, collision rate, utilization rate and channel selection ratio. The simulation results show that the system throughput of cross-domain heterogeneous fusion network is increased by more than 60%, system channel utilization efficiency is increased by 20%, and the collision rate of service packets is maintained at 10%, which verifies the adaptability of the algorithm to different business scenarios.

Key words: heterogeneous converged network, air, space and ground integration, deep reinforcement learning, spectrum utilization rate, access protoco, fair scale, dimensioning, system throughput

CLC Number:

TN92

XIAO Xiong, LIU Hongyan, YI Mengjie. Heterogeneous Converged Network Access Algorithm Based on Deep Reinforcement Learning[J].Electronic Science and Technology, 2024, 37(11): 7-12.

Figures/Tables 6

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

References 18

[1]	Kaytaz U, Ucar S, Akgun B, et al. Distributed deep rein-forcement learning with wideband sensing for dynamic spectrum access[C]. Seoul:IEEE Wireless Communications and Networking Conference, 2020:1-6.
[2]	Kassab R, Destounis A, Tsilimantos D, et al. Multi-agentdeep stochastic policy gradient for event based dynamic spectrum access[C]. London:IEEE the Thirty-first Annual International Symposium on Personal,Indoor and Mobile Radio Communications, 2020:1-6.
[3]	Zhong C, Lu Z, Gursoy M C, et al. A deep actorcritic reinforcement learning framework for dynamic multi-channel access[J]. IEEE Transactions on Cognitive Communications and Networking, 2019, 5(4):1125-1139.
[4]	Xu Y, Yu J, Buehrer R M. The application of deep reinforcement learning to distributed spectrum access indynamic heterogeneous environments with partial observations[J]. IEEE Transactions on Wireless Communications, 2020, 19(7):4494-4506.
[5]	Chen S, Li Y. An overview of robust reinforcement learning[C]. Nanjing:IEEE International Conference on Networking,Sensing and Control, 2020:1-6.
[6]	Zhu Z, Liu Y, Canova M. Energy management of hybrid electric vehicles via deep Q-networks[C]. Denver:American Control Conference, 2020:3077-3082.
[7]	Bensalem M, Dizdarević J, Jukan A. DNN placement and inference in edge computing[C]. Opatija:The Fourty-third International Convention on Information,Communication and Electronic Technology, 2020:479-484.
[8]	Kwadjo D T, Mbongue J M, Bobda C. Performance exploration on preimplemented CNN hardware accelerator on FPGA[C]. Maui:International Conference on Field-Programmable Technology, 2020:298-299.
[9]	Liu W, Zeng Y. Motor imagery tasks EEG signals classification using ResNet with multi-time-frequency representation[C]. Xi'an:The Seventh International Conference on Intelligent Computing and Signal Processing, 2022:2026-2029.
[10]	Routray N, Rout S K, Sahu B. Breast cancer prediction using deep learning technique RNN and GRU[C]. Gunupur:The Second International Conference on Computer Science,Engineering and Applications, 2022:1-5.
[11]	余琼芳, 牛冬阳. 基于LSTM网络的矿山压力时空混合预测[J]. 电子科技, 2023, 36(2):67-72.
	Yu Qiongfang, Niu Dongyang. Mixed prediction of mine pressure time and space based on LSTM network[J]. Electronic Science and Technology, 2023, 36(2):67-72.
[12]	Ye X, Yu Y, Fu L. Multi-channel opportunistic access for heterogeneous networks based on deep reinforcement learning[J]. IEEE Transactions on Wireless Communications, 2022, 21(2):794-807.
[13]	Yu Y, Liew S C, Wang T. Non-uniform time-step deep Q-network for carrier-sense multiple access in hetero-geneous wireless networks[J]. IEEE Transactions on Mobile Computing, 2021, 20(9):2848-2861.
[14]	谢羿. 基于北斗全球传输网络的天基测控任务研究[D]. 长沙: 国防科技大学, 2018:1-46.
	Xie Yi. Research on space-based TT&C task based on Beidou global transmission network[D]. Changsha: National University of Defense Technology, 2018:1-46.
[15]	刘凯. 基于NB-IOT的接入优化关键技术研究[D]. 厦门: 厦门大学, 2019:1-52.
	Liu Kai. Research on key technologies of access optimization based on NB-IOT[D]. Xiamen: Xiamen University, 2019:1-52.
[16]	党凯强. 基于LoRa和北斗短报文的森林防火探测终端设计[D]. 太原: 中北大学, 2021:1-63.
	Dang Kaiqiang. Design of forest fire detection terminal based on LoRa and Beidou short message[D]. Taiyuan: North University of China, 2021:1-63.
[17]	赵一蒙. 移动自组网多跳传输优化技术研究[D]. 北京: 中国电子科技集团公司电子科学研究院, 2022:1-62.
	Zhao Yimeng. Research on multihop transmission optimization technology for mobile ad hoc network[D]. Beijing: China Academic of Electronics and Information Technology, 2022:1-62.
[18]	Nishio T, Yonetani R. Client selection for federated learning with heterogeneous resources in mobile edge[C]. Shanghai:IEEE International Conference on Communications, 2019:1-7.

Heterogeneous Converged Network Access Algorithm Based on Deep Reinforcement Learning

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 18

Related Articles 2

Metrics

Comments

Recommended 0

[1]	TANG Chao, ZHANG Fan. Motion Planning of Manipulator Based on Improved SoftActor-Critic Algorithm [J]. Electronic Science and Technology, 2024, 37(11): 47-54.
[2]	ZHANG Haitao,ZHANG Tong,ZHANG Yuhui,GUAN Yinfeng,ZHANG Fengdeng. Task Partitioning Optimization Algorithm Based on MrsP Protocol [J]. Electronic Science and Technology, 2023, 36(3): 36-41.