Deep Reinforcement Learning Based Safety-critical Scenario Generation for Autonomous Ship Digital Testing

Abstract

The development of Maritime Autonomous Surface Ships (MASS) represents a transformative shift in maritime transportation, where digital testing systems play a pivotal role in evaluating autonomous behaviors under complex and safety critical conditions. Such systems, incorporating software simulation, scenario libraries and agent–environment interaction, are essential for addressing challenges arising from environmental complexity and rare but high risk events. While existing research predominantly focuses on collision avoidance scenarios from the own ship perspective, few studies investigate adversarial or attacking ship behaviors that are crucial for stress testing MASS intelligence. This study introduces an end to end framework for generating safety critical scenarios from the attacking ship perspective using deep reinforcement learning (DRL). Real world trajectory and geographic data from the Strait of Singapore are leveraged to initialize realistic interactions, covering head on, overtaking and crossing maneuvers. Scenario evaluation integrates collision related indicators such as Time to Collision (TTC) and risk metrics derived from the Closest Point of Approach (CPA) to assess scenario severity and behavioral fidelity. Experimental results show that the proposed framework produces diverse and high quality critical scenarios that enhance the realism, challenge level and robustness of MASS digital testing systems. Overall, this work provides an AI-driven scenario generation methodology that enriches digital testing environments and supports the development of more reliable and resilient autonomous vessel systems.