Visual perception with object detection for autonomous driving under complex weather conditions

Abstract

The commercialization of autonomous driving relies heavily on reliable environmental perception in all-weather and all-scenario conditions. Complex weather such as low-light, rain, fog, snow, and haze severely degrades the perception performance of visual sensors and LiDAR, becoming a core bottleneck restricting the robustness of object detection models. This paper systematically reviews the research progress of visual perception and 2D/3D object detection for autonomous driving in complex weather, and constructs a full-link optimization paradigm from data, feature, model, and deployment levels. At the data level, a hybrid style transfer augmentation method based on physical priors and generative AI is proposed to solve the problems of scarce complex weather samples and imbalanced distribution. At the feature level, a weather-adaptive attention mechanism and a multi-modal feature alignment module are designed to alleviate feature degradation and noise interference under low-light and rainy-foggy conditions. At the model level, a cross-modal fusion detection framework with 2D visual guidance and 3D point cloud geometric constraints is built to achieve accurate target positioning in all weather. At the deployment level, quantization distillation and dynamic inference strategies are proposed to balance accuracy and real-time performance on vehicle-mounted platforms. Extensive experiments on KITTI, NuScenes, and a self-built complex weather dataset show that the proposed framework improves 2D and 3D object detection accuracy by 9.3% and 7.8%, respectively, compared with baseline models, while keeping inference delay within 40 ms. Finally, this paper deeply analyzes the limitations of current technologies and prospects future research directions such as extreme weather perception, self-supervised learning, and multi-sensor collaboration, providing a systematic theoretical reference and engineering practice guide for the development of allweather perception systems for autonomous driving.