ROSE: Roadside Oversight-Guided Scenario Enhancement with Self-Supervised Coupling for multi-modal Perception

Abstract

With the rapid advancement of autonomous driving and Intelligent Transportation Systems (ITS), roadside perception—an essential component of vehicle-to-everything (V2X) communication—has become a critical foundation for large-scale traffic monitoring and data-driven safety decisions. However, under adverse environmental conditions such as rain, snow, fog, and nighttime, current roadside perception systems often face limited training data, high annotation costs, and significant performance degradation due to poor model robustness. To tackle these challenges without relying on additional labeled data, this paper introduces ROSE (Roadside Oversight-guided Scenario Enhancement), a unified closed-loop framework designed to enhance the resilience and adaptability of roadside multi-modal perception systems. ROSE integrates three key components: (1) RISA, a first-order physics-guided cross-modal augmentation module that generates physically plausible and semantically aligned adverse-weather samples; (2) SSL Coupling, a cross-modal self-supervised learning network that facilitates robust feature alignment; and (3) a curriculum scheduling mechanism guided by vision-language models (VLM) to adaptively prioritize learning difficulty. Experimental results show that ROSE achieves competitive detection accuracy while providing improved robustness and cross-modal consistency across adverse weather scenarios, demonstrating strong generalization potential for roadside multi-modal perception under challenging environmental conditions. These findings suggest that ROSE offers a practical and effective pathway toward building resilient V2X cooperative perception systems.