Traffic Signal Control via Proximal Policy Optimization with Reward Shaping to Minimize Waiting Time and Violations

Abstract

Traffic signal control at intersections is a key challenge in urban traffic management, particularly when accounting for non-compliant driver behavior such as red-light violations. While Proximal Policy Optimization (PPO) has shown promise for adaptive traffic control, most implementations overlook the stochastic nature of such violations, limiting real-world applicability. This study proposes an enhanced PPO-based traffic signal control approach that incorporates reward shaping to minimize vehicle waiting time and traffic violations jointly. The method modifies the reward function by adding a fixed bonus when no violations occur and a logarithmically scaled penalty when violations are detected. Experiments were conducted in Simulation of Urban Mobility (SUMO) using a real-world intersection model, with aggressive and violation-prone driver behavior generated through domain randomization. Evaluation covered three training scenarios (5%, 10%, and 20% violation rates) and two additional test scenarios with different intersection layouts. In training scenarios, PPO with reward shaping reduced violations to 3-4 while maintaining moderate delays of 14-22 seconds, outperforming PPO without reward shaping, which primarily reduced delays but failed to improve compliance. In unseen scenarios, the proposed method consistently reduced delays, while gains in compliance varied with traffic conditions. These results show that integrating violation-sensitive reward shaping into PPO enables policies that minimize both waiting time and violations, offering a practical and robust approach for intelligent traffic signal control in complex urban environments.