In this paper, we introduce a novel hierarchical decision-making framework that integrates control barrier functions (CBFs) with reinforcement learning (RL) to enhance the safety and efficiency of autonomous vehicle merging at on-ramps. The proposed technique is comprised of three layers: a behavioral planning layer that employs proximal policy optimization (PPO) algorithm aims to learn its driving behaviors, which are not necessarily compliant with vehicle dynamics; a motion control layer that adopts model predictive control (MPC) framework to ensure the kinematic feasibility to follow the trajectories with heading angles and velocities provided by the upper layer; and lastly, a control barrier function-guided layer, which is the core innovation of this paper, is implemented. It leverages the forward invariance of control barrier functions to design a safety controller that refines the output of MPC to maintain continuous driving safety. The refined results are then used as objective function, guiding subsequent updates in the learning process. Simulation results demonstrate that the proposed CBF-guided architecture significantly improves training efficiency and performance, achieving a 60% reduction in training time and 6% increase in success rate compared to standard RL methods.
Control Barrier Function-Guided Deep Reinforcement Learning for Decision-Making of Autonomous Vehicle at On-Ramp Merging
IEEE Transactions on Intelligent Transportation Systems ; 26 , 6 ; 8919-8932
2025-06-01
7404072 byte
Article (Journal)
Electronic Resource
English
Deep reinforcement learning algorithm based ramp merging decision model
SAGE Publications | 2025
|RACE-SM: Reinforcement Learning Based Autonomous Control for Social On-Ramp Merging
ArXiv | 2024
|