Behavior decision-making of intelligent vehicles using budgeted reinforcement learning

Abstract

Autonomous vehicles (AVs) commonly adopt overly conservative driving behaviors to reduce the likelihood of low-probability traffic accidents. However, such conservatism often leads to ineffective interaction with other road users and may even result in prolonged inactivity. To address this issue, this paper proposes a hierarchical framework for behavior decision-making and motion planning that explicitly accounts for the dynamic balance between safety and driving efficiency. First, a risk budget is incorporated into a Markov Decision Process (MDP) framework to design a budgeted reinforcement learning–based decision policy. By independently constructing the cost constraint and reward function, the AV can flexibly adjust its acceptable risk level and make optimal driving decisions that maximize cumulative rewards under predefined safety constraints. Then, a polynomial-sampling-based motion planning algorithm in the Frenet frame is developed to translate high-level decisions into feasible reference trajectories, which are executed through a low-level trajectory tracking controller to update the AV’s states in the environment. Finally, simulation results demonstrate that the proposed behavioral planning method achieves a highly comparable safety-efficiency trade-off frontier to constrained MDP benchmarks but under a single unified network framework. This eliminates the necessity of deploying multiple independent models, and enables online risk budget adjustment to dynamically realize a spectrum of driving styles ranging from conservative to aggressive.