Aug 2026· Future Internet· Vol 18, pp. 427· 0 citations· 20 references
TL;DR
A semantic-aware autonomous navigation framework implemented on a ROS 1 (Melodic) mobile robot equipped with a two-dimensional light detection and ranging (LiDAR) sensor and an RGB-D camera that offers a low-cost, extensible basis for language-guided robots in smart environments.
Abstract
Autonomous mobile robots require robust perception and high-level reasoning to operate in complex indoor environments. While the Robot Operating System (ROS) provides a modular framework for mapping and navigation, classical pipelines lack semantic understanding and natural-language interaction. This paper presents a semantic-aware autonomous navigation framework implemented on a ROS 1 (Melodic) mobile robot equipped with a two-dimensional light detection and ranging (LiDAR) sensor and an RGB-D camera. The system integrates LiDAR-based simultaneous localization and mapping (SLAM), the ROS navigation stack (move_base), and a lightweight You Only Look Once (YOLO) object detector for real-time on-board perception, and it anchors detections into the metric map to build a semantic map. A large language model (LLM) interprets natural-language instructions and converts them into structured navigation goals. Perception and control run entirely on-board the Jetson Nano, whereas the LLM is invoked episodically as a cloud service, yielding a hybrid embedded/cloud architecture. In indoor trials over a semantic map of two object classes, the system grounded all ten multilingual commands to the correct objects at a reasoning cost of about one second, and safely rejected a command referring to an unmapped object. We present this as an in-depth single-platform case study: owing to the 4 GB memory budget, the on-board detector and the full navigation stack are time-multiplexed rather than run continuously in parallel; nonetheless, a single degraded end-to-end trial confirmed that perception, online semantic anchoring, language grounding, and navigation compose within one continuous session. The framework offers a low-cost, extensible basis for language-guided robots in smart environments.
Natural language interaction provides an intuitive way for non-expert users to communicate with robotic platforms. However, transforming user requests into executable navigation actions remains a challenging task, requiring the integration of language understanding, environment perception, and autonomous navigation. This work presents a language-driven navigation framework that enables mobile robots to interpret user requests in natural language to move the robot to a destination and autonomously navigate towards it. The framework is composed of modular ROS 2 components that cooperate to transform natural language instructions into navigation actions. Given a natural language request referring to a target in the environment (e.g.,"go to the mail box"), the system identifies the referenced object, estimates its position using RGB-D data, and generates a navigation goal, which is then executed through the ROS 2 Nav2 navigation stack. The ROS 2-based implementation facilitates portability across different robotic platforms, requiring only the configuration of the corresponding topics and services. The system is evaluated in both simulation and real-world scenarios using a TurtleBot3 Waffle and a Unitree Go2 robot with a RealSense camera. Experimental results show that the framework successfully interprets both direct commands and contextual requests, generates meaningful natural-language feedback, and navigates towards the desired target. These results demonstrate the feasibility of combining semantic perception and autonomous navigation to provide an intuitive human-robot interaction paradigm. Code will be released as open source upon acceptance.
Jose Mart'inez-Fajardo, Pablo Pueyo, F. Caballero et al.· 0 citations
Traditional automated guided vehicles (AGVs) are restricted by their reliance on predefined paths, limiting adaptability in dynamic warehouse environments. While autonomous mobile robots (AMRs) overcome this limitation through on-board simultaneous localization and mapping (SLAM) and autonomous navigation, standard configurations often suffer from top mounted-sensor blind spots when loads are carried on the chassis. To address these coverage gaps, an indoor logistic AMR based on the robot operating system 2 (ROS2) was designed and evaluated. The platform was developed by combining a multi-LiDAR perception stack with low-cost industrial actuation and a lightweight, fleet-style user interface. Within the system architecture, data from two light detection and ranging (LiDAR) sensors were merged at the topic level into a single virtual scan for SLAM toolbox and Nav2. Additionally, actuation and wheel odometry were driven by an RS-485 Modbus-based brushless DC (BLDC) motor controller, while ultrasonic sensors for short-range safety, an inertial measurement unit (IMU) for orientation, and a network of microcontroller calling stations communicating via message queuing telemetry transport (MQTT) were integrated into the platform. Experimental validation demonstrated successful multi-LiDAR fusion, with the Modbus motor driver achieving a motion-control error of 0.36 % and a speed-retrieval error of 0.43 %. Furthermore, calling-station commands were reliably executed over MQTT, and a point-to-point navigational repeatability of 10.3 cm was achieved. These findings indicate that an integrated multi-LiDAR ROS2 AMR provides a highly practical solution for indoor logistics. Through the proposed sensor merger and calling-station handshake, two recurring vulnerabilities of standard ROS2 deployments—single-LiDAR coverage gaps and Nav2 goal-overwriting behavior—were successfully resolved.
Leonard P. Rusli, Michael Jonathan, Rusman Rusyadi· Journal of Mechatronics, Ele...· 0 citations
Autonomous semantic indoor navigation requires robust low-level control and high-level understanding of objects and spatial context in cluttered and partially occluded environments. While deep reinforcement learning (DRL) methods such as twin delayed deep deterministic policy gradient (TD3) enable reactive obstacle avoidance, they typically struggle with long-horizon semantic navigation, where object-location memory and language-level reasoning are required. We present a lightweight hierarchical two-stage framework that, to the best of our knowledge, is introduced for the first time to integrate a locally deployed vision–language model (VLM), semantic object coordinate memory, and a TD3-based DRL controller for language-conditioned indoor navigation. In Stage 1, the robot performs semantic exploration using odometry, 2D LiDAR, and VLM-based object recognition to build a geometric map and store detected object categories with their estimated world coordinates in a structured javaScript object notation (JSON) semantic memory. In Stage 2, a natural language query is used to retrieve the target object coordinates from memory and pass them to a TD3 target point navigation policy, which performs mapless navigation using odometry and RealSense RGB-D perception. The proposed framework combines open-vocabulary VLM-based object coordinate estimation, LiDAR mapping, RGB-D perception, and language grounding within a unified semantic memory representation. Experiments in a ROS-integrated realistic simulation demonstrate consistent goal-reaching performance and improved navigation efficiency compared with an Artificial Potential Field baseline using the same VLM and a DRL + GPT-4o mini configuration. We also compare the proposed VLM-based recognition module with YOLO-World v2.6 and Grounding DINO, showing that the VLM-based approach provides more reliable semantic grounding and target-coordinate estimation in the tested indoor navigation scenarios, particularly for flexible natural language object queries.
Yernar Akhmetbek, Ayaulym Parmash, Temirlan Meiramkhanov et al.· Italian National Conference...· 0 citations
Foundation-model-based vision-language navigation (VLN) has advanced autonomous robot navigation by enabling robots to interpret natural-language instructions, identify semantic goals, and follow user-specified behavioral rules. However, existing VLN systems rely heavily on cloud-hosted foundation models for language understanding and semantic grounding, limiting their applicability where network connectivity is unavailable and reliable metric goal localization is required. Although recent small language models (SLMs) enable fully onboard inference, their suitability for navigation instruction decomposition has not been systematically evaluated. This paper makes three contributions toward fully onboard VLN for outdoor environments. First, we present the first systematic benchmark of 17 edge-deployable SLMs against 4 online APIs for robotic navigation instruction decomposition, evaluating accuracy and latency on human-annotated instructions across three computing platforms and providing practical guidance for selecting onboard language models. Second, we propose a lightweight hybrid semantic-geometric goal localization framework that combines open-vocabulary object detection, prompted segmentation, and LiDAR geometry to estimate metric goals, while maintaining visual bearing guidance when reliable geometric observations are unavailable. Third, we integrate these advances into Edge-BehAV, a fully onboard extension of the BehAV architecture that enables cloud-independent behavior-guided navigation. Experimental results show that the best offline SLM matches the instruction decomposition performance of the strongest cloud API while running approximately 9x faster and without network connectivity. The proposed goal localization framework reduces mean goal-distance error from 2.05 m to 0.20 m at lower computational cost, and the complete system succeeds in 31 of 32 closed-loop outdoor trials.
Ali Salmasi, Xianjia Yu, Tomi Westerlund· 0 citations
For mobile robots to execute commands given by humans in environments lacking localization infrastructure, they must be capable of making semantic inferences about the environment. To this end, we present a task-based navigation method for a mobile robot using single monocular camera. The robot’s perception mechanism consists of a SSD-InceptionV2 for target object detection and MiDaS-Small for depth map estimation. By exploiting the geometric advantage of the motion plane of ground robots, metric distances derived from relative depth outputs are used in the obstacle avoidance algorithm, and a Kalman filter is applied to ensure continuity in target object tracking. Tests conducted on a Jetson TX2 platform verified that the system exhibits stable performance at approximately 15 FPS and successfully completes its task. The results demonstrate that mobile robot navigation systems can be developed in infrastructure-free environments using solely camera-based perception.
M. Özmen, Şevval Akpınar, Furkan Dağdevir et al.· Signal Processing and Commun...· 0 citations
Collaborative robotics is a representative task-oriented 6G use-case, where communication quality should be reflected in mission execution, environment understanding, and closed-loop operation rather than packet delivery alone. This demo paper presents a robot-edge semantic communication (SemCom) testbed integrating robot-side visual compression, edge-side semantic mapping, and dashboard-based mission interaction. A mobile robot equipped with RGB-D sensing and LiDAR runs ROS 2, while a Jetson Orin edge node performs reconstruction, RTAB-Map mapping, semantic object handling, and browserbased visualization. As an initial proof of concept, RGB frames are encoded on the robot into VQ-VAE tokens using an ONNX Runtime encoder and reconstructed on the edge using a PyTorch decoder. A 320 X 240 image is represented by an 80 X 60 token grid with a packed payload of 5400 bytes, corresponding to a 42.67X reduction relative to model-input RGB bytes. The reconstructed visual stream is further associated with depth, pose, and 3D mapping information to generate a semantic map for downstream robotic applications. The demo exposes the full path from semantic visual transport to object-level map interaction, and provides a practical platform for future task-aware 6G networking studies at the intersection of SemCom, embodied AI, and physical AI-enabled robotics. A video of the demo is available at https://tinyurl.com/Tos09
Peizheng Li, Xinyi Lin, Sajida Gufran et al.· 0 citations
