CPNav is introduced, a navigation framework that combines collision-aware planning with statistically calibrated VLM action scoring for map-free object-goal navigation and provides finite-sample guarantees at a user-specified risk level.
Abstract
Directing a robot to locate a target object in unseen environments remains a central challenge in embodied AI. Vision-Language Model (VLM)-based methods provide rich semantic reasoning but often produce uncalibrated and overconfident action decisions. We introduce CPNav, a navigation framework that combines collision-aware planning with statistically calibrated VLM action scoring for map-free object-goal navigation. From RGB-D input, CPNav constructs a navigability map, generates collision-free motion primitives, and queries two VLMs for goal detection and action scoring. The resulting scores are calibrated offline via conformal prediction (CP) using oracle actions obtained through a tree-structured exploration procedure with backtracking, providing finite-sample guarantees at a user-specified risk level. The CP layer is model-agnostic and can wrap any VLM policy. Online, calibrated VLM scores guide action selection, while a tree-structured exploration history records alternative actions and allows backtracking. We evaluate CPNav in both simulation and real-world hardware settings. In AI Habitat simulation, compared with the uncalibrated baseline method, CPNav achieves relative SR and SPL improvements of 6.2% and 11.4% on HM3D v0.2, and 12.4% and 8.8% on MP3D, respectively. In real-world deployment on a modified Hiwonder MentorPi robot, the corresponding improvements in SR and SPL are 24.9% and 15.2%, respectively. Code is available at https://github.com/practice-lab-ucla/CPnav.
NaViRRator, a framework that translates user-specified start and goal locations on such maps into navigation instructions for a pretrained vision-and-language navigation (VLN) policy, is presented, supporting route-grounded language as a modular interface between human-readable maps and pretrained navigation policies.
Ayun Lee, Jiseon Kim, Giseop Kim· arXiv.org· 1 citation
Real-world mobile robots may not always have access to complete or accurate prior maps. In such cases, vision-language models (VLMs), which combine visual perception with semantic reasoning, offer a promising way to interpret environmental changes and revise navigation decisions. This study develops and evaluates two V...
Brian Fitzpatrick, Noorbakhsh Amiri Golilarz, Soroush Korivand· International Conference on...· 0 citations
Robots manipulating in cluttered spaces must often plan from partial views where foreground objects occlude the target, surrounding geometry, and usable free space. Reactive policies often miss these hidden constraints, while exhaustive search becomes computationally prohibitive in dense clutter. We present SCOUT, an o...
Evaluations across multiple environments and object- and relation-goal tasks show improved closed-loop navigation reliability and reduced language-model token consumption with capacity-controlled evidence exposure under the evaluated inference configuration.
General-purpose service robots need navigation systems that can handle diverse human requests in unfamiliar environments, combining task generality with scene generality. Some existing methods fine-tune multimodal large language models (MLLMs) to predict navigation actions, making their behavior dependent on the covera...
Jin-Kai Zhang, Jing-Yi Xu, Yuan-Hong Yu et al.· 3 citations
A vision-language-action policy for autonomous lunar rover docking and its performance relative to a modular navigation pipeline is designed and a scalable training framework is developed that generates safety-aware expert demonstrations in a high-fidelity lunar simulation environment.
Si-Tan Huang, G. Gao· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduJul 30, 2026
From feet to fingertips — we are teaching robots intelligent whole-body control, fine dexterity, and teamwork to complete a broad range of complex tasks.