Abstract:

The majority of existing beamforming algorithms rely heavily on the quality of instantaneous channel state information (CSI),which is unsuitable for practical systems and ignores power control issues,resulting in serious inter-user interference and lowering the spectrum efficiency.A deep reinforcement learning-based joint beamforming and power control technique is proposed to jointly tackle the beamforming design and power control problems without the requirement for perfect CSI.First,an information exchange protocol is proposed to facilitate the base station to understand environmental information,and a dual-model system with a centralized training distributed execution structure is designed to solve the joint optimization problem.The cloud utilizes Deep Q-Learning (DQN) to design the beamforming after receiving the local samples uploaded by the base station,which is collected and uploaded to the cloud by the base station.Considering that deep Q learning is not applicable to continuous variables,we employ the deep deterministic strategy gradient algorithm(DDPG) to solve the power control problem.The cloud model is broadcast to all base stations for distributed execution in order to acquire local samples once the training is completed.Simulation results show that the proposed scheme for spectral efficiency optimization significantly outperforms the traditional beamforming algorithm.

Key words: deep reinforcement learning, deep deterministic policy gradient, beamforming, power control