A systematic analysis of the machine learning and deep learning models underpinning vehicle autonomy, spanning classical convolutional neural networks for object detection and semantic segmentation to recurrent and Transformer-based architectures for trajectory prediction and motion planning is presented.
Abstract
Autonomous driving systems rely on a sophisticated pipeline of artificial intelligence models to perceive, predict, and plan in dynamic environments. This review presents a systematic analysis of the machine learning and deep learning models underpinning vehicle autonomy, spanning classical convolutional neural networks (CNNs) for object detection and semantic segmentation to recurrent and Transformer-based architectures for trajectory prediction and motion planning. It also provides a critical examination of the autonomous vehicle sensor stack, including cameras, LiDAR, radar, ultrasonics, and GNSS/IMU as data acquisition systems, highlighting modality-specific AI challenges such as monocular depth estimation, 3D point cloud processing, and radar Doppler interpretation. The evolution of perception and decision-making pipelines is reviewed, contrasting modular architectures with end-to-end learning paradigms that directly map raw sensor data to control commands, and discussing their trade-offs in interpretability, safety assurance, and robustness to rare edge cases. We further survey specialized hardware accelerators and heterogeneous automotive SoCs designed to meet stringent real-time and power constraints. Industrial strategies are compared, including multi-modal sensor fusion and vision-centric approaches based on large-scale imitation learning. Finally, we identify open challenges related to robustness under adverse conditions, domain shift, causal ambiguity, and the need for interpretable and certifiable AI in safety-critical autonomous driving systems.
Sensor fusion plays a critical role in enabling reliable perception for autonomous driving systems by integrating heterogeneous data from multiple sensors such as cameras, LiDAR, radar, and inertial units. However, inconsistencies in spatial alignment, temporal synchronisation, and data representation present significant challenges to achieving a unified understanding of the driving environment. This paper proposes a conceptual framework that interprets sensor fusion as a reconstructive process, transforming diverse sensory inputs into a coherent environmental model. The study systematically analyses three levels of fusion: data-level, feature-level, and decision-level, and examines how artificial intelligence enhances each stage through learned alignment, cross-modal feature representation, and uncertainty-aware decision making. Furthermore, the paper connects fusion strategies to key autonomous driving tasks, including object detection, tracking, localisation, and planning, highlighting the relationship between task requirements and fusion architecture design. Finally, major challenges such as domain shift, long-tail scenarios, sensor failure, and interpretability are discussed. This work provides a structured perspective on AI-driven sensor fusion and its role in building robust and adaptive intelligent driving systems.
The paradigm shift to "vehicle-road synergy" infrastructure as a critical mechanism for providing the safety redundancy and edge-computing capabilities needed for fully reliable, next-generation autonomous driving systems is explored.
Xichen Huang· Applied and Computational En...· 0 citations
Autonomous driving has become a transformative technology poised to reshape modern transportation systems. This paper explores multimodal fusion techniques that integrate various sensors with machine vision for autonomous driving. We examine the integration of various sensor modalities, including cameras, LiDAR, and millimeter-wave radar, alongside advanced machine vision algorithms such as YOLO, Faster R-CNN, Point Pillars, and MVX-Net for environment perception. This work addresses major challenges in sensor fusion, including data synchronization, coordinate transformation, real-time computation and conflict resolution of heterogeneous sensor data. We systematically analyze three typical fusion architectures: data-level, feature-level and decision-level fusion, and compare their performance in information retention, computational efficiency and system robustness. Through representative application cases in object detection and classification, high-precision localization and mapping, and decision-making and path planning, we demonstrate how multimodal fusion significantly enhances the robustness, accuracy, and reliability of autonomous vehicle perception systems. The paper further discusses current limitations including computational overhead, adverse-weather robustness, and lack of standardized evaluation, and outlines future directions such as end-to-end learning, 4D radar integration, and V2X-enabled cooperative perception. The results prove that reliable multimodal fusion is a core prerequisite for realizing safe and stable autonomous driving.
Yu-He Lu· Applied and Computational En...· 0 citations
Interaction with the physical world differentiates physical AI from other forms of AI. Autonomous driving exemplifies this; vehicles must perceive and respond to dynamic environments with human-like or better perception-reaction times. This survey addresses the fundamental challenge of deploying high-performance models for multimodal fusion in resource-constrained automotive environments. We organise state-of-the-art deep learning approaches into five paradigms—CNN-based, transformer-based, dense BEV-based, sparse-based, and hybrid—revealing trade-offs in accuracy, latency, and efficiency, as well as strengths and limitations in robustness under adverse operational design domains. The hardware-aware perspective is a differentiating contribution, presenting strategies for deployment on automotive platforms, reducing inference latency by up to 50% and improving robustness in adverse conditions by up to 20%. By synthesising sensor fusion, deep learning, compute platforms, and hardware-awareness, this work equips researchers and practitioners with actionable insights and strategies for perception systems, bridging theoretical advances and production-grade autonomous driving requirements.
Ken Power, M. Halton, Ciarán Eising· IEEE Open Journal of Vehicul...· 0 citations
The proposed framework comprises sensor fusion, image processing, feature extraction, deep neural network inference and explainability mechanisms such as Gradient-weighted Class Activation Mapping, Local Interpretable Model-Agnostic Explanations and SHapley Additive exPlanations.
Johan Håstad's mentor Arne Andersson, Börje Langefors· International Journal of Eme...· 0 citations
Autonomous robots play a crucial role in industrial manufacturing, healthcare, transportation, logistics, agriculture, disaster response, planetary exploration, and service robotics. Reliable visual perception is essential for enabling robots to recognize objects, understand scenes, localize themselves, and navigate safely in dynamic environments. Although CNN-based vision models have significantly improved perception accuracy, they often struggle to capture long-range dependencies and generalize to complex or unseen environments. Recent advances in Transformer-based vision models address these limitations by employing self-attention mechanisms to learn both local visual features and global contextual relationships. Architectures such as Vision Transformer (ViT), Swin Transformer, DETR, SAM, and Mask2Former have achieved remarkable performance in object detection, semantic segmentation, SLAM, localization, obstacle avoidance, and autonomous navigation. This paper presents a comprehensive review and proposes the Transformer-Based Visual Perception Models for Autonomous Robots (TBVPM-AR) framework. The framework integrates RGB cameras, depth sensors, LiDAR, IMUs, multimodal sensor fusion, transformer-based feature extraction, contextual reasoning, and edge-cloud computing to achieve robust perception in dynamic environments. Mathematical formulations for self-attention, positional encoding, and feature embedding provide the theoretical foundation of the architecture. Experimental evaluations demonstrate that the proposed framework outperforms CNN-based and hybrid approaches on standard robotic perception benchmarks, achieving over 98% visual perception accuracy with improved scene understanding, localization, obstacle detection, navigation, and computational efficiency. The proposed architecture offers a scalable, explainable, and adaptable solution for future Industry 5.0, collaborative robotics, autonomous vehicles, and smart cyber-physical systems.
Mahabala H.N· International Journal of Int...· 0 citations