An LLM-driven hierarchical room-to-object (HRO) framework for zero-shot object-goal navigation, which guides the agent to explore and navigate to the target object in a coarse-to-fine manner, underscoring LLMs'strong potential for zero-shot object-goal navigation.
Abstract
Zero-shot object-goal navigation aims to enable an intelligent agent to explore and navigate to objects of unknown categories in an unfamiliar environment without specific target training. In zero-shot navigation tasks, pre-trained large models are usually employed to leverage their prior knowledge for guiding the agent's navigation. However, existing zero-shot object-goal navigation methods based on large language models (LLMs) merely utilize LLMs as flat reasoning tools to directly associate objects or regions. They lack the hierarchical spatial cognition modeling of human-like room semantics to object localization, which leads to strong blindness in exploration, insufficient accuracy in semantic association, and failure to fully unleash the common-sense reasoning potential of LLMs. This paper proposes an LLM-driven hierarchical room-to-object (HRO) framework for zero-shot object-goal navigation, which guides the agent to explore and navigate to the target object in a coarse-to-fine manner. Experiments on Gibson and HM3D datasets verify that our HRO framework achieves superior success rate and generalization over existing LLM-based methods, underscoring LLMs'strong potential for zero-shot object-goal navigation.
Universal zero-shot goal-oriented navigation requires an agent to locate object categories, target instances, or text-described goals in unseen environments without task-specific policy training. Existing methods usually rely on either broad semantic exploration or fine-grained target verification, but they can become inefficient when semantic cues are sparse, ambiguous, or insufficient for reliable graph matching. This paper presents ForesightNav, a training-free universal zero-shot navigation framework that adaptively combines context-aware semantic value mapping, graph-based target verification, and TSP-based frontier ordering. Guided by semantic-score dispersion and graph-matching confidence, the planner switches among geometric exploration, semantic exploration, partial-match exploration, and target verification, balancing broad goal-directed search with fine-grained target confirmation. Experiments on MP3D, HM3D, and RoboTHOR across Object Navigation (ObjectNav), Instance Navigation (InstanceNav), and Text Navigation (TextNav) show favorable performance across different goal modalities. Compared with UniGoal, ForesightNav improves ObjectNav performance on three evaluated benchmarks, with the largest gains on RoboTHOR ObjectNav: 18.6 percentage points in Success Rate (SR) and 11.4 in Success weighted by Path Length (SPL). On HM3D, it achieves 65.3% SR on InstanceNav and 29.6% SR on TextNav, suggesting that the same training-free planning process remains applicable to instance- and language-specified goals. Foundation-model analysis and ablation results further support the effectiveness of adaptive switching, context-aware semantic mapping, and TSP-based frontier ordering. Code is available at https://anonymous.4open.science/r/ForesightNav
How do humans navigate to a target object in an unmapped, unseen environment? We certainly do not wander aimlessly. Instead, human explorers naturally rely on spatial context, leveraging the inherent co-occurrence of everyday objects to infer a target’s probable location. However, conventional zero-shot object navigation methods construct a value map using only the target object, directing the agent toward the frontier with the highest score. This paper proposes a framework that leverages contextual cues from a Large Language Model (LLM). The proposed method uses an LLM to infer co-occurring objects near the target object. Based on current observations, the exploration direction is toward the frontiers where both the target and its co-occurring objects are most likely to be found. To compute the similarity between the target object and co-occurring objects, we designed value map fusion methods using static, dynamic, and adaptive structures. We verified the generalizability of our approach by integrating it into existing value-map based zero-shot object navigation models. Evaluations on the HM3D dataset demonstrated that the proposed method improves the Success Rate and the Success weighted by Path Length relative to the baselines. Therefore, mimicking human reasoning processes with LLM-provided contextual cues, we successfully enhanced zero-shot object navigation performance.
Sangmin Park, Minhwan Ko, Kyoobin Lee· 2026 23rd International Conf...· 0 citations
This paper presents a hierarchical fast-slow agent that turns what the robot has already seen into the object of deliberation in zero-shot object-goal navigation, and reaches the highest success rate among the zero-shot methods compared here.
Zhaochen Lan, Zhi Yang, Yuxiang Fu et al.· 0 citations
Mobile manipulation requires a robot to navigate to a target object or receptacle and then perform intended manipulation. However, reaching the vicinity of the target does not guarantee a manipulation-ready base pose, a problem known as last-mile navigation. Prior methods for last-mile navigation either rely on manual pose annotation or task-specific training, limiting their scalability to open-vocabulary settings with fine-grained spatial constraints. We propose UniLM-Nav, a unified framework for zero-shot open-vocabulary last-mile navigation. UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose reasoning, all resolved with a shared multimodal large language model (MLLM) backend. Specifically, UniLM-Nav first selects a reference view that best captures the target object or receptacle from recently collected observations. It then grounds task-relevant affordance point in the selected view and lifts the result into the robot-centric coordinate frame. Finally, conditioned on the grounded affordance, task context, and robot geometry, it infers a manipulation-ready base pose for the robot. We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points. Analyses show that the components of our method are crucial to final performance, and that the choice of MLLM also has a substantial effect. We further deploy UniLM-Nav on a Unitree B2 quadruped robot with a 6-DoF Unitree Z1 manipulator, validating its applicability to real-world mobile manipulation tasks.
Zhuofan Zhang, Tianxu Wang, Guoxi Zhang et al.· 0 citations
Vision-and-language navigation (VLN) enables robots to follow instructions in previously unseen environments. Recently, a training-free paradigm has emerged: the robot queries a multimodal LLM to understand its observations and plan the next action. However, long-horizon navigation based on either image streams or dense map inevitably introduces a growing memory and reasoning bottleneck. We present HAM-VLN, a decision-coupled, agent-authored memory that equips the robot with a persistent, depth-grounded world graph. In the same model call used to select the next action, HAM-VLN also records semantic and reflective information---including room type, objects, navigation progress, and failure notes. Recent waypoints remain verbatim within a bounded window, while older history re-enters the context only through retrieval scored by relevance, recency, and salience, together with one-hop topological expansion. This design requires no additional LLM calls beyond the per-waypoint decision. Compared to previous methods, HAM-VLN not only improves various navigation metrics but also reduces the context length by more than 65%. Specifically, HAM-VLN achieves 61.0% Success Rate (SR) on VLN-CE R2R, 52.7% SR on VLN-CE RxR, and 79.7% SR on HM3D-v2 ObjectNav without any training.
An Liu, Bingxi Liu, Hongyu Ding et al.· 0 citations
Autonomous semantic indoor navigation requires robust low-level control and high-level understanding of objects and spatial context in cluttered and partially occluded environments. While deep reinforcement learning (DRL) methods such as twin delayed deep deterministic policy gradient (TD3) enable reactive obstacle avoidance, they typically struggle with long-horizon semantic navigation, where object-location memory and language-level reasoning are required. We present a lightweight hierarchical two-stage framework that, to the best of our knowledge, is introduced for the first time to integrate a locally deployed vision–language model (VLM), semantic object coordinate memory, and a TD3-based DRL controller for language-conditioned indoor navigation. In Stage 1, the robot performs semantic exploration using odometry, 2D LiDAR, and VLM-based object recognition to build a geometric map and store detected object categories with their estimated world coordinates in a structured javaScript object notation (JSON) semantic memory. In Stage 2, a natural language query is used to retrieve the target object coordinates from memory and pass them to a TD3 target point navigation policy, which performs mapless navigation using odometry and RealSense RGB-D perception. The proposed framework combines open-vocabulary VLM-based object coordinate estimation, LiDAR mapping, RGB-D perception, and language grounding within a unified semantic memory representation. Experiments in a ROS-integrated realistic simulation demonstrate consistent goal-reaching performance and improved navigation efficiency compared with an Artificial Potential Field baseline using the same VLM and a DRL + GPT-4o mini configuration. We also compare the proposed VLM-based recognition module with YOLO-World v2.6 and Grounding DINO, showing that the VLM-based approach provides more reliable semantic grounding and target-coordinate estimation in the tested indoor navigation scenarios, particularly for flexible natural language object queries.
Yernar Akhmetbek, Ayaulym Parmash, Temirlan Meiramkhanov et al.· Italian National Conference...· 0 citations