A Vision language model (VLM)-assisted semantic-mapping pipeline is demonstrated that extracts obstacles and environmental context from satellite imagery, nautical charts, and forecast Application Programming Interface (API) instead of onboard sensors, reaching 96% navigability accuracy as a drop-in replacement for hand-specified obstacle geometry.
Abstract
Large language models can turn a natural-language mission into a sequence of robot actions, but they do not have a sense of physics: they cannot judge how long a command should run, or whether it will make the robot drift into an obstacle. We proposed the use of a world model to expand the capabilities of Large Language model-based planners. Our method has three components: a physics-grounded neural world model, a three-phase gradient-based trajectory optimizer, and a Model Predictive Controller (MPC)-style closed-loop replanner with a trust-region guard. The language model decides what to do, and the world model decides how long, whether that means driving eight thrusters through 6 DOF or two differential thrusters through 3 DOF. We evaluate two marine vehicle classes operating near offshore wind infrastructure: a 6-DOF Autonomous Underwater Vehicle (AUV) and a 3-DOF differential-drive Autonomous Surface Vehicle (ASV). In five benchmark missions per platform, both vehicles reach every goal with zero predicted collisions, and both transfer to GazeboSim under ocean current, waves, and thruster dynamics, remaining collision-free and cutting GazeboSim goal-distance error versus the ungrounded baseline by 70-82% (ASV) and roughly 93% (AUV), after a residual fine-tuning pass that separately reduces surrogate rollout Root Mean Square Error (RMSE) by 60% (AUV) and 69% (ASV). For the ASV we further demonstrate a Vision language model (VLM)-assisted semantic-mapping pipeline that extracts obstacles and environmental context from satellite imagery, nautical charts, and forecast Application Programming Interface (API) instead of onboard sensors, reaching 96% navigability accuracy as a drop-in replacement for hand-specified obstacle geometry.
A simple and effective approach to apply test-time scaling to VLN for UAV navigation through an iterative refinement process that requires no extra model training, guiding the model to re-evaluate its initial navigation plan for better accuracy and safety.
Feinan Cheng, Dongliang Xu, Wenli Nong et al.· 0 citations
A novel framework, LLM-Augmented Semantic Reasoning for Robust AUV Docking (DockLLM), which pioneers the integration of multi-modal large models’ semantic generalization capabilities into the underwater docking task and maintains a robust 89% success rate under simulated heavy occlusion scenarios.
Bolun Zhang, Canjun Yang· Machine Vision and Applicati...· 0 citations
SLM-A* is designed as a language-native planning agent that can interoperate with multi-agent LLM frameworks as a callable planning sub-module, thus providing an architectural capability not available to conventional planners that operate on numerical graph representations.
This paper addresses autonomous intervention with an underwater vehicle--manipulator system (UVMS) in confined, cluttered, and partially known environments, where poor maneuverability, narrow passages, and uncertain execution may cause the robot to enter unrecoverable regions. We propose MANTA, a three-layer hierarchical planning-and-control framework that couples passage accessibility, manipulation feasibility, and closed-loop execution. The first layer performs global connectivity reasoning in a conservative reduced base space to extract traversable corridor candidates toward the task region. The second layer refines each candidate corridor by jointly optimizing the continuous base motion and arm trajectory, producing a collision-free base--arm trajectory. The third layer learns a reach-and-hold base policy using Gaussian-process model-based reinforcement learning (MBRL) through MC-PILCO, enabling trajectory tracking and station keeping at the planned manipulation state. During execution, the framework monitors map updates and can trigger recovery and route repair when the active passage becomes infeasible. MANTA is evaluated in confined UVMS planning and closed-loop tracking experiments. Across 120 matched planning queries, it achieves higher task success than full-state sampling-based baselines while producing larger clearance margins and lower arm motion. The learned MC-PILCO policy further reduces position and yaw tracking errors on both training and unseen tube-like references. These results show MANTA as a structured and data-efficient framework for safe autonomous underwater intervention in caves, tubes, and cluttered subsea structures.
Mohamed Abdelwahab, Ruggero Carli, Damiano Varagnolo et al.· 0 citations
Ship path planning is a central challenge in autonomous navigation for unmanned surface vehicles and maritime autonomous surface ships. It is not simply a shortest-path problem, but a constrained sequential decision process that must reconcile collision risk, route efficiency, COLREGs compliance, vessel dynamics, and environmental uncertainty. Here we review the field through a unified framework based on planning scope, decision basis, and deployment requirements. We examine search- and sampling-based, geometric and rule-based, optimization-based, learning-driven, and hybrid methods, with particular emphasis on deep reinforcement learning for discrete decisions, continuous maneuvering, multi-vessel interaction, and safety-oriented control. Representative studies are compared across objective and reward design, state representation, exploration and policy optimization, rule integration, disturbance modeling, simulation platforms, and operational validation. The synthesis identifies persistent barriers, including ambiguous rule formalization, partial observability, strategic coupling among vessels, inconsistent benchmarks, limited cross-scenario generalization, and insufficient full-scale validation. We further discuss priority directions in explicit safety constraints, digital twins, transfer and meta-learning, world models, scalable multi-agent coordination, and large-model-assisted mission reasoning. We argue that progress will depend less on further algorithmic proliferation than on integrated, verifiable architectures that combine data-driven adaptation with model-based structure, standardized evaluation, and staged real-world assurance.
Weijun Wang, Mingjie Li, Bushuo Wang et al.· Journal of Marine Science an...· 0 citations
Reliable and accurate trajectory prediction is critical for the safe and efficient operation of unmanned aerial vehicles (UAVs) in complex urban environments, where flight dynamics are subject to wind disturbances, dense obstacle fields, and strongly phase-dependent behavior. Conventional physics-based approaches are limited by parameter uncertainty, simplifying assumptions, and unmodeled disturbances, while data-driven models may lack physical plausibility and robustness across different flight phases. To address these limitations, this paper proposes a physics-guided hybrid residual-correction framework for UAV trajectory prediction. The approach combines a physics-based model with a data-driven sequence model and learns a residual correction that compensates for the deviation between analytical prediction and observed flight behavior. In addition, the model is trained with physics-guided feasibility regularization to promote realistic speed, acceleration, jerk, and landing descent behavior. Experimental evaluation on a real-world test set shows that the proposed method yields improved results relative to the stand-alone physics model, the LSTM model, and an MLP-based fusion model across all major metrics, including RMSE, MAE, ADE, and FDE. Phase-wise analysis further demonstrates strong improvements in cruise and landing, while highlighting takeoff as the most challenging phase. The results indicate that combining physical structure with learned residual correction provides a more accurate, physically consistent, and operationally interpretable approach for UAV trajectory forecasting.
Md Ashraful Islam, Stanley Förster, Tianxiong Zhang et al.· Scientific Reports· 0 citations