Jul 2026· IEEE Internet of Things Journal· pp. 1-1· 1 citation· 53 references
Computer Science
TL;DR
A large language model (LLM) enhanced differentiable trajectory planning framework for IoT-enabled autonomous driving is proposed and a surrounding agent centric data augmentation strategy is introduced to reorganize sur rounding agent trajectories as additional planning supervision, thereby improving the training distribution without collecting additional raw data.
Abstract
Autonomous driving planning is a key component of IoT-enabled intelligent transportation systems, requiring vehicles to generate safe, efficient, and executable trajectories in complex urban environments from multi-source contextual information. While imitation learning (IL) has shown promise on large-scale datasets, IL-based planners still suffer from limited coverage of complex long-tail interactions, weak consistency with downstream constrained refinement, and insufficient use of high level scene semantics under real time constraints. To address these issues, this paper proposes a large language model (LLM) enhanced differentiable trajectory planning framework for IoT-enabled autonomous driving. Specifically, we introduce a surrounding agent centric data augmentation strategy to reorganize sur rounding agent trajectories as additional planning supervision, thereby improving the training distribution without collecting additional raw data. We further design a complexity-aware asyn chronous LLM-based semantic enhancement module to extract scene-related high-level semantic features with controlled online overhead. In addition, a differentiable optimization module is incorporated to refine generated trajectories with explicit residual penalties while backpropagating optimization gradients to the upstream planner. Experiments show that the proposed method achieves the best overall scores of 83.63 and 78.29 on the nuPlan closed-loop nonreactive and reactive Hard20 benchmarks, respectively, and CARLA-ROS tests further verify its online deployment and real time closed-loop execution capability.
Autonomous driving in complex urban environments requires trajectory planning that balances safety, efficiency, and human-like behavior. Although imitation learning (IL) can capture expert driving patterns from large-scale demonstrations, existing IL-based planners still face challenges in safety-critical scenarios and long-tail traffic distributions. Meanwhile, optimization-based planners provide explicit constraint handling but are often separated from upstream learning modules, limiting their ability to jointly improve trajectory generation and planning feasibility. To address these issues, we propose a hybrid trajectory planning framework that integrates IL-based multimodal trajectory proposal with differentiable optimization. In the proposed framework, an IL backbone generates candidate ego trajectories and surrounding-agent predictions, while a differentiable optimizer refines the selected trajectory using multi-objective cost functions with learnable weights related to safety, efficiency, and comfort. This design enables optimization objectives and constraints to provide gradient feedback to the upstream planning network, improving the consistency between candidate generation and downstream planning objectives. In addition, we introduce a surrounding agent centric data augmentation strategy that reuses real-world trajectories of surrounding vehicles as additional expert demonstrations, thereby enriching complex interaction and long-tail scenarios without extra data collection. Closed-loop experiments on the nuPlan benchmark show that the proposed method achieves a composite score of 94.04, outperforming PLUTO’s 93.14 while using only 30% of the training data. The results demonstrate that the proposed framework improves closed-loop planning performance, trajectory feasibility, and data efficiency under complex urban driving scenarios.
Shihao Zhang, Ziyu Song, Zhaochen Xia et al.· Proceedings of the Instituti...· 0 citations
An efficient Mamba-based feature extraction framework for jointly encoding vehicle trajectories and map information is proposed and achieves superior performance in terms of minADE, minFDE, and minMR, while maintaining high computational efficiency.
J. Li, L. Wang, J. Pei· Revista Internacional de Mét...· 0 citations
Large-scale language models for autonomous driving enable enhanced global understanding and long-horizon planning. However, when deployed in isolated vehicles, limited sensing range and occlusions restrict reliable decision-making, and the substantial computational and latency overhead makes on-board deployment impractical. Cooperative driving provides a potential solution by leveraging external agents for information exchange, but existing methods remain limited in semantic reasoning capability under practical constraints. To address these challenges, we propose DH-VLM, a dual-horizon cooperative latent reasoning framework that enables asymmetric semantic cooperation between the infrastructure and ego vehicle. The infrastructure aggregates multi-layer hidden states to form a global-reasoning horizon latent guidance, which is integrated into the ego model through an Infrastructure-Driven Latent Evolution mechanism for conditional latent refinement. This enables the ego vehicle to leverage long-range contextual understanding while preserving autonomous decision-making within its local planning horizon. Furthermore, we construct a cooperation-oriented question-answer (QA) dataset covering fundamental scene understanding and ego-personalized comprehension to support counterfactual and safety-aware reasoning. Extensive experiments demonstrate that DH-VLM achieves state-of-the-art planning performance, outperforming the previous state of the art by 14.6% in L2 error and 26.9% in collision rate. Compared with query-based end-to-end cooperative driving methods, our approach reduces the communication cost by 57.3% and GPU memory usage by 25.5%, while maintaining strong robustness against infrastructure guidance errors, providing a practical and robust paradigm for cooperative autonomous driving.
A vision-based end-to-end autonomous parking framework trained through imitation learning that introduces a historical context fusion encoder to capture temporal dependencies from past vehicle motions, a dual-stream attention decoder to enhance interaction between scene features and trajectory representations, and kinematic-aware auxiliary losses to enforce smooth and feasible trajectory generation.
Daisy X. M. Zheng, Bingli Zhang, Xinyu Wang et al.· Engineering Research Express· 0 citations
A comprehensive Explainable AI (XAI) evaluation framework is introduced, including temporal sensitivity analysis, interaction-aware perturbation studies, spatial influence analysis, and gradient-based feature attribution methods that provide insights into how the model captures temporal motion dependencies, neighboring vehicle interactions, and environmental context during trajectory prediction.
R. Megalingam, Naveen Prasaad Selvarajan, Pritty Vijay· Italian National Conference...· 0 citations
Recent research in machine and deep learning has shown the potential of learningbased motion planning approaches to improve the driving behavior of automated vehicles, especially in complex environments. However, their complex nature and lack of transparency can hinder explainability and trustworthiness and complicate safety assurance. Motivated by these challenges, we propose a hybrid planning architecture that combines the advantages of machine learning with the verifiability and the determinism of classical approaches. Specifically, we developed a deep neural network to interpret complex traffic scenes and propose driving behavior, while an optimization-based supervision layer validates this proposal and enforces explicit drivability and safety constraints. We evaluate the learned planner's driving behavior in open-loop studies on real-world urban data, discuss system integration aspects for stable closed-loop operation, and report results from real-world deployment on our research vehicle karl..
Jean-Pierre Busch, Guido Linden, Jan Bergmann et al.· 0 citations
