空天地融合车载网络场景下的资源优化策略

doi:10.19665/j.issn1001-2400.20250405

摘要/Abstract

摘要：

空天地融合车载网场景下,无人机设备由于电池容量和能源有限,无法为任务卸载提供长期有效支持;低轨卫星受资源成本以及通信延迟、时延抖动的影响难以为大规模车联网任务提供稳定的高带宽通信服务。针对空天地融合车载网络场景下无人机和低轨卫星的资源优化问题,提出了一种基于多任务深度强化辅助学习(Multi-Task Deep Reinforcement and Auxiliary Learning,MTDRAL)的任务卸载以及功率调整、缓存决策的方案。首先构建了任务切分与传输模型、时延模型、能耗模型、服务器计算与缓存模型和问题模型;然后,基于对任务处理时延、服务器能耗以及缓存命中率的综合考虑,给出了基于MTDRAL的任务卸载及资源调度方案;最后将所提方案与随机卸载策略方案、成功率贪婪决策方案、基于柔性动作-评价算法的多网络深度强化学习的卸载方案、基于深度确定性策略梯度算法的多网络深度强化学习的卸载方案进行了对比实验。实验结果表明:所提方案在服务器数量为14、车载终端数量为10时,综合得分相较于4种对比方案,分别领先约134.41%,31.32%,38.93%,29.49%;所提方案具有较好的性能,能更好地满足空天地融合车载网场景下的任务卸载需求。

关键词: 空天地融合车载网络, 资源优化, 任务卸载, 强化学习

Abstract:

In the scenario of the space-air-ground integrated vehicle network (SAGVN),due to the limited battery capacity and energy,unmanned aerial vehicle (UAV) devices cannot provide long-term effective support for task offloading; due to resource costs,communication delays,and delay jitter,it is difficult for low-Earth orbit (LEO) satellites to provide stable high bandwidth communication services for the Internet of Vehicles(IoV) tasks.To address the resource optimization problem of UAVs and LEO satellites in the scenario of SAGVN,a solution for task offloading,power adjustment,and caching decision-making based on multi-task deep reinforcement and auxiliary learning (MTDRAL) is proposed.First,the task segmentation and transmission model,latency model,energy consumption model,server computation and caching model,and problem model are established.Then,by comprehensively considering the task processing delay,server energy consumption,and cache hit rate,an MTDRAL-based solution for task offloading and resource scheduling is proposed.Finally,comparative experiments are conducted between the proposed solution and four benchmark strategies:a random offloading strategy,a success-rate-based greedy decision strategy,a multi-network deep reinforcement learning offloading strategy based on the soft actor-critic algorithm,and a multi-network deep reinforcement learning offloading strategy based on the deep deterministic policy gradient algorithm.Experimental results show that when the server count is 14 and the number of vehicle terminals is 10,the proposed solution outperforms the four comparative strategies by 134.41%,31.32%,38.93%,and 29.49% in comprehensive scoring,respectively.The proposed solution has a good performance and can better meet the task offloading requirements in the scenario of SAGVN.

Key words: space-air-ground integrated vehicle network(SAGVN), resource optimization, task offloading, reinforcement learning

中图分类号:

TP929.5

朱思峰, 黄长龙, 宋兆威, 张宗辉, 朱海, 乔蕊. 空天地融合车载网络场景下的资源优化策略[J]. 西安电子科技大学学报, 2025, 52(3): 134-149.

ZHU Sifeng, HUANG Changlong, SONG Zhaowei, ZHANG Zonghui, ZHU Hai, QIAO Rui. Research on resource optimization strategy in the scenarios of the space-air-ground integrated vehicle network[J]. Journal of Xidian University, 2025, 52(3): 134-149.

图/表 15

图1

图2

表1

图3

表2

图4

表3

图5

图6

图7

图8

图9

图10

图11

图12

参考文献 30

[1]	CAI Q, ZHOU Y, LIU L, et al. Prioritized Assignment with Task Dependency in Collaborative Mobile Edge Computing[J]. IEEE Transactions on Mobile Computing, 2024, 23(12):13505-13521.
[2]	PENG Y, TANG X, ZHOU Y, et al. Computing and Communication Cost-Aware Service Migration Enabled by Transfer Reinforcement Learning for Dynamic Vehicular Edge Computing Networks[J]. IEEE Transactions on Mobile Computing, 2024, 23(1):257-269.
[3]	QI Y, ZHOU Y, LIU Y F, et al. Traffic-Aware Task Offloading Based on Convergence of Communication and Sensing in Vehicular Edge Computing[J]. IEEE Internet of Things Journal, 2021, 8(24):17762-17777.
[4]	崔亚平, 石宏吉, 吴大鹏, 等. 内容新鲜度保障的车联网多智能体缓存分发策略[J]. 通信学报, 2025, 46(1):52-66. doi: 10.11959/j.issn.1000-436x.2025013
	CUI Yaping, SHI Hongji, WU Dapeng, et al. Multi-Agent Caching and Distribution Strategy with Content Freshness Guarantee for Internet of Vehicles[J]. Journal on Communications, 2025, 46(1):52-66. doi: 10.11959/j.issn.1000-436x.2025013
[5]	王朝炜, 杜嘉楠, 王程, 等. 软件定义卫星网络中基于业务调度的混合路由算法[J]. 电子学报, 2024, 52(5):1506-1515. doi: 10.12263/DZXB.20221090
	WANG Zhaowei, DU Jianan, WANG Cheng, et al. Hybrid Routing Algorithm Based on Service Scheduling in Software-Defined Satellite Networks[J]. Acta Electronica Sinica, 2024, 52(5):1506-1515.
[6]	王博文, 郑建, 孙彦景, 等. 应急场景无人机自组网部分重叠信道动态分配方法[J]. 电子与信息学报, 2024, 46(12):4373-4382.
	WANG Bowen, ZHENG Jian, SUN Yanjing, et al. Dynamic Allocation Method of Partially Overlapping Channels in UAV Ad Hoc Networks for Emergency Scenarios[J]. Journal of Electronics & Information Technology, 2024, 46(12):4373-4382.
[7]	李博, 王改芳, 杨洪娟, 等. 利用离散时间聚合图的无人机编队最短时延路由协议[J]. 电子与信息学报, 2024, 46(5):1931-1939.
	LI Bo, WANG Gaifang, YANG Hongjuan, et al. Minimum Delay Routing Protocol for UAV Formation Based on Discrete-Time Aggregation Graph[J]. Journal of Electronics & Information Technology, 2024, 46(5):1931-1939.
[8]	BAI L, LIU J, WANG J, et al. Data Aggregation in UAV-Aided Random Access for Internet of Vehicles[J]. IEEE Internet of Things Journal, 2022, 9(8):5755-5764.
[9]	WANG L L, GUI J S, DENG X H, et al. Routing Algorithm Based on Vehicle Position Analysis for Internet of Vehicles[J]. IEEE Internet of Things Journal, 2020, 7(12):11701-11712.
[10]	HAN R, GUAN Q, YU F R, et al. Congestion and Position Aware Dynamic Routing for the Internet of Vehicles[J]. IEEE Transactions on Vehicular Technology, 2020, 69(12):16082-16094.
[11]	LIU R, LIU A, QU Z, et al. An UAV-Enabled Intelligent Connected Transportation System with 6G Communications for Internet of Vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(2):2045-2059.
[12]	ALSADIE D. Efficient Task Offloading Strategy for Energy-Constrained Edge Computing Environments:A Hybrid Optimization Approach[J]. IEEE Access, 2024, 12(1):85089-85102.
[13]	CUI Y, LI H, ZHANG D, et al. Multiagent Reinforcement Learning-Based Cooperative Multitype Task Offloading Strategy for Internet of Vehicles in B5G/6G Network[J]. IEEE Internet of Things Journal, 2023, 10(14):12248-12260.
[14]	XIAO H, XU C, MA Y, et al. Edge Intelligence:A Computational Task Offloading Scheme for Dependent IoT Application[J]. IEEE Transactions on Wireless Communications, 2022, 21(9):7222-7237.
[15]	YU G, HE Y, WU J, et al. Mobility-Aware Proactive Edge Caching for Large Files in the Internet of Vehicles[J]. IEEE Internet of Things Journal, 2023, 10(13):11293-11305.
[16]	HOU C, ZHOU C, HUANG Q, et al. Cache Control of Edge Computing System for Tradeoff Between Delays and Cache Storage Costs[J]. IEEE Transactions on Automation Science and Engineering, 2024, 21(1):827-843.
[17]	ARAF S, SAHA A S, KAZI S H, et al. UAV Assisted Cooperative Caching on Network Edge Using Multi-Agent Actor-Critic Reinforcement Learning[J]. IEEE Transactions on Vehicular Technology, 2023, 72(2):2322-2337.
[18]	胡玉姣, 黄韬, 贾庆民, 等. 通算存智一体协同的未来网络模型[J]. 通信学报, 2024, 45(5):12-28. doi: 10.11959/j.issn.1000-436x.2024089
	HU Yujiao, HUANG Tao, JIA Qingmin, et al. Future Network Architecture of Integrated Computing,Communication,Storage and Intelligence[J]. Journal on Communications, 2024, 45(5):12-28. doi: 10.11959/j.issn.1000-436x.2024089
[19]	MIAO J, WANG Z, NING X, et al. A UAV-Assisted Authentication Protocol for Internet of Vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(8):10286-10297.
[20]	LI Y L, CHAI Z Y, TAN F, et al. Temporal Data Scheduling in Internet of Vehicles Using an Improved Decomposition-Based Multi-Objective Evolutionary Algorithm[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(5):5282-5295.
[21]	张华, 隋敬麒, 武鹏飞, 等. 5G无线通信场景需求与技术演进分析[J]. 长江信息通信, 2021, 34(7):197-199.
	ZHANG Hua, SUI Jingqi, WU Pengfei, et al. Analysis on Requirements and Technological Evolution of 5G Wireless Communication Scenarios[J]. Changjiang Information & Communications, 2021, 34(7):197-199.
[22]	NDIKUMANA A, TRAN N, HO T, et al. Joint Communication,Computation,Caching,and Control in Big Data Multi-Access Edge Computing[J]. IEEE Transactions on Mobile Computing, 2020, 19(6):1359-1374.
[23]	VARGHESE N V, MAHMOUD Q H. A Hybrid Multi-Task Learning Approach for Optimizing Deep Reinforcement Learning Agents[J]. IEEE Access, 2021, 9(1):44681-44703.
[24]	ZHENG K, JIANG G, LIU X, et al. DRL-Based Offloading for Computation Delay Minimization in Wireless-Powered Multi-Access Edge Computing[J]. IEEE Transactions on Communications, 2023, 71(3):1755-1770.
[25]	AHMADIAN A, SHIN W, PARK H. Long-Term Throughput Maximization in Wireless Powered Communication Networks:A Multitask DRL Approach[J]. IEEE Internet of Things Journal, 2024, 11(11):19616-19631.
[26]	TARIQ M N, WANG J, RAZA S, et al. Toward Optimal Resource Allocation:A Multi-Agent DRL Based Task Offloading Approach in Multi-UAV-Assisted MEC Networks[J]. IEEE Access, 2024, 12(1):81428-81440.
[27]	FU F, KANG Y, ZHANG Z, et al. Soft Actor-Critic DRL for Live Transcoding and Streaming in Vehicular Fog-Computing-Enabled IoV[J]. IEEE Internet of Things Journal, 2021, 8(3):1308-1321.
[28]	CHEN Y, SUN Y, WANG C, et al. Dynamic Task Allocation and Service Migration in Edge-Cloud IoT System Based on Deep Reinforcement Learning[J]. IEEE Internet of Things Journal, 2022, 9(18):16742-16757.
[29]	LIU J, ZHANG H. Power Allocation in Ultra-Dense Networks Through Deep Deterministic Policy Gradient[J]. IEEE Wireless Communications Letters, 2022, 11(12):2502-2506.
[30]	ZHANG H, LIU R, KAUSHIK A, et al. Satellite Edge Computing with Collaborative Computation Offloading:An Intelligent Deep Deterministic Policy Gradient Approach[J]. IEEE Internet of Things Journal, 2023, 10(10):9092-9107.

终端服务类型	卸载频率	延迟要求	用户体验
自动驾驶	高	高	<10 ms
辅助驾驶	高	高	<10 ms
车联网	高	高	<10 ms
路况信息同步	高	中	31~50 ms
远程控制车辆	中	中	250~1 100 ms
语音通话	高	低	0.2~2 s
车辆数据采集	低	低	<10 s

卷积网络	卷积核大小	滑动步长
Conv1	1×2	2
Conv2	1×6	6
Conv3	1×2	2

参数	值	参数	值	参数	值
B/MHz	50	D_K/MB	30	A_S	4×4
w/dBm	10^-3	T_Z	[0.1,1]	O_r	4
P_i/W	40	T_D/MB	[0.2,2]	高敏感时延范围/ms	[5,10]
α	4	T_L/MB	[0.2,1]	中敏感时延范围/ms	[10,50]
P_K/W	240	F_R/帧	100	低敏感时延范围/ms	[50,200]
单个晶体管的功率/nW	1	每个服务器中晶体管数量	10⁸	一个任务所拥有的子任务的数量	[5,10]