Diffusion-Based Reinforcement Learning for Multi-Objective Adaptive Control in Autonomous Car-Following

Abstract

This paper investigates adaptive multi-objective control for autonomous car-following using model predictive control (MPC). Conventional MPC with predefined cost-function parameters has limited adaptability in balancing safety and comfort under varying traffic conditions. To address this limitation, an intent-driven adaptive MPC framework is proposed. A long shortterm memory network is first employed to infer the driving intent of the preceding vehicle. Then, a modified twin delayed deep deterministic policy gradient (TD3) algorithm is developed, in which the conventional multilayer perceptron (MLP) actor is replaced by a diffusion policy actor. The diffusion actor generates a continuous action that adaptively configures the MPC cost weights. Simulation results on real-world NGSIM trajectories show that the proposed method achieves ride comfort comparable to that of the conventional MPC baseline. In emergency braking, the proposed method improves safety regulation over the MPC baseline. Compared with the MLP-based TD3 baseline, the proposed method produces smoother control responses, demonstrating the effectiveness of the diffusion-based actor for adaptive MPC.