DynaFuser: Uncertainty-Aware Dynamic Multimodal Fusion for End-to-End Autonomous Driving
In the field of end-to-end autonomous driving, extreme visual degradation environments, such as nighttime and low-light conditions, often lead to a sharp decline in sensor perception capabilities, thereby triggering safety risks like trajectory deviation. Under such scenarios, significant feature distribution uncertainty and spatial divergence arise between the camera and LiDAR modalities. Traditional closed-box fusion mechanisms, lacking explicit physical constraints, are highly susceptible to failure in Out-of-Distribution (OOD) scenarios. To address this issue, this paper proposes DynaFuser, an uncertainty-aware multimodal fusion architecture designed for dynamic and complex environments. The architecture introduces a Collaborative Dual-stream Adaptive Feature Refinement Mechanism (CDA-FRM) to perform direction-aware enhancement on visual textures and LiDAR geometric structures, respectively. The core contribution lies in proposing a Heterogeneous Modal Dynamic Entropy Weight (HM-DEW) fusion strategy based on non-extensive statistical mechanics. This strategy utilizes variable-order Tsallis entropy to quantify intra-modal uncertainty in real-time and employs Kullback-Leibler (KL) divergence to achieve cross-modal consistency alignment. Furthermore, to address trajectory deviation caused by perceptual perturbations, the model constructs a Dynamic Channel Decoupling Waypoint Prediction Mechanism (DCD-WPM), which integrates asymmetric feature reconstruction with a spatio-temporal gated autoregressive decoder to significantly improve the geometric consistency and temporal coherence of trajectory outputs. Evaluated in the CARLA closed-loop simulation environment, DynaFuser achieves state-of-the-art performance on both the Longest6 and LAV benchmarks, obtaining a Driving Score (DS) of 60.45% and 67.42%, respectively, along with a Route Completion (RC) of 94.33% and 93.24%, thereby demonstrating superior scene adaptability and generalization capability. Note to Practitioners—Autonomous driving in real-world traffic remains highly challenging due to dynamic lighting conditions, unpredictable obstacles, and sensor degradation under adverse environments. In practice, engineers frequently encounter challenges when fusing data from cameras and LiDAR due to their distinct characteristics and occasional misalignments, which can lead to unsafe trajectory predictions. This work introduces DynaFuser, a dynamic multimodal fusion framework designed to improve trajectory planning reliability in complex environments. The core concept involves adaptively refining camera and LiDAR features separately, subsequently combining them through a dynamic weighting strategy that accounts for environmental conditions, such as low illumination or dense traffic. By doing so, the system significantly reduces trajectory errors and improves route completion in challenging scenarios, as demonstrated in CARLA closed-loop benchmarks. For industry practitioners, this approach provides a more robust trajectory planning baseline, particularly for autonomous driving fleets operating in urban areas where lighting and occlusion frequently vary. Our current validation is limited to simulation, and real-world deployment will require further testing with large-scale on-road datasets. Moreover, computational efficiency remains a critical factor when integrating such models into automotive-grade processors. Future work may extend this framework to incorporate additional modalities (e.g., radar, V2X communication) and further optimize inference speed. Beyond autonomous vehicles, the proposed fusion strategy could also benefit other robotic applications requiring reliable navigation under uncertain sensor conditions.