关键词: 车辆跟随控制, 深度强化学习, 离线训练, 分层控制, 比例积分微分控制

Abstract:

The current car following control methods based on the control theory rely on models for both car speed and distance,which suffer from a lack of generalization and achieve stable and smooth control results with difficulty.To address this problem,we propose a data-efficient hierarchical car following control method that does not depend on car kinematic models.The upper layer of the proposed method constructs a dataset based on the perception results of car coordinates,speed,and other onboard sensors.A deep reinforcement learning model is trained to perform car following,avoiding the reliance on prior knowledge and eliminating the need for real-world training.Training samples are randomly selected from the dataset,which improves data utilization.The lower layer of the method implements real-time control of the car acceleration and angular velocity using a PID controller,which avoids the control jitter caused by the instability of deep reinforcement learning policies,resulting in smoother control.To verify the performance of the algorithm,both simulation and real-world experiments are conducted.Experimental results show that the proposed algorithm can keep the distance between the following car and the target car within a safe and reasonable range.The comparative experiments demonstrate that the proposed algorithm achieves more stable,smoother,and safer car following control in both lateral and longitudinal directions.

Key words: car-following control, deep reinforcement learning, offline training, hierarchical control, proportion integral differential(PID) control