Reinforcement Learning-Based Dynamic Merging Optimization for Highway Work Zones Considering Driver Compliance Heterogeneity

Abstract

To address the severe safety challenges facing Highway Work Zones (HWZs) in mixed traffic flow environments, particularly the stochastic disturbances introduced by human “Driver Compliance Heterogeneity,” this paper proposes a macroscopic dynamic merging control strategy based on the Proximal Policy Optimization (PPO) algorithm. This study formulates the dynamic merging optimization problem as a Markov Decision Process (MDP), constructing a comprehensive state space that integrates lane-specific traffic states, vehicle composition, driver compliance heterogeneity, and real-time conflict counts. Through coordination with upstream VMS, the PPO agent learns a near-optimal control policy to dynamically adjust the merge point location in response to real-time traffic fluctuations. Within the Simulation of Urban Mobility (SUMO) simulation environment, the proposed strategy is rigorously benchmarked against traditional static merging (Early/Late) strategies, a Poisson regression-based dynamic strategy, and a Deep Q-Network (DQN) baseline. Experimental results demonstrate that the PPO algorithm excels in balancing safety and efficiency, yielding a significant reduction in traffic conflicts compared to other control methods. Further analysis reveals that the strategy effectively suppresses the generation of shockwaves through effective control in high-risk scenarios characterized by low compliance rates, validating its immense potential for proactive risk mitigation in complex work zone scenarios. The analysis confirms that the agent learns to establish an adaptive “Safety Buffer” by proactively shifting merge points upstream in low-compliance conditions. This finding validates the potential of deep reinforcement learning to dynamically trade off spatial resources for system stability in mixed-autonomy bottlenecks.