Jul 2026· Signal Processing and Communications Applications Conference· pp. 1-4· 0 citations· 9 references
Abstract
This study comparatively examines the task-oriented grasping problem, which is of critical importance in robotic manipulation, through modern Vision-Language Models. Within the scope of the study, the performance rates of the GraspMolmo model, specifically trained for robotic tasks, and Gemini ER-1.5, a general-purpose multimodal AI, were analyzed. The evaluation process was conducted using the TaskGrasp-Image dataset, which encompasses a wide range of objects and tasks, through natural language commands and RGB-D images. The accuracy of the grasping coordinates generated by the models was systematically assessed across varying tolerance thresholds, revealing that both models achieved high accuracy rates. The robotics-specialized model demonstrated a notable advantage over the general-purpose model, particularly under strict tolerance conditions. Error analysis showed that the majority of failed predictions targeted functionally incorrect regions of the object rather than falling outside it entirely, indicating that semantic reasoning rather than geometric localization constitutes the primary challenge.
Experimental results in various task scenarios show that the proposed framework consistently improves overall task success rates compared with unimodal settings with different LLMs and achieves a higher success rate compared to using only visual or force data.
Transparent objects such as glass containers, test tubes, and plastic bottles are common in robotic manipulation scenarios, but their refractive and reflective surfaces produce incomplete RGB-D geometry and make task-specific grasp selection unreliable. This paper presents an integrated vision-language system for transparent object perception and task-oriented grasping. First, we construct VLM-DRE, a transparent object image instruction dataset with 12,700 images and 38,100 image-instruction-bounding-box triplets. LoRA fine-tuning of Molmo-7B improves target click accuracy from 86.4% to 91.5% and IoU@0.75 from 57.5% to 69.1%. Second, MSR-Net performs monocular depth completion and mask prediction using multi-scale adaptive feature fusion and progressive feature refinement, achieving RMSE 0.066, mAP 98.61%, and IoU 94.12% on Syn-TODD, and RMSE 0.118, mAP 99.02%, and IoU 87.95% on ClearPose. Third, LMF-Net combines RGB-D cross-modal fusion with learnable multi-factor matching to rank AnyGrasp 6-DoF candidates, reaching 77.8% Top-1 and 90.5% Top-3 accuracy on TaskGrasp-Image and improving PRISM-Real success from 61.1% to 68.5%. On a RealSense D435i–Unitree Z1 Pro platform, the complete system obtains 85.4% success with manual clicks and 71.3% with VLM-predicted clicks, supporting perception-to-grasping integration while highlighting target localisation and runtime as deployment bottlenecks.
Kejian Ni, Xiepeng Yang, Tao Chen et al.· Robotics· 0 citations
Generalist robots need to perform diverse tasks while operating in dynamic, uncertain, and unstructured environments, often around human beings. Vision-language-action (VLA) models have recently emerged as a promising and flexible framework for integrating perception, reasoning, robotic control, and action execution to develop generalist robotic policies. This systematic literature review (SLR) examines more than 140 VLA-related publications between 2020 and 2025 following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. To the best of our knowledge, it is the first PRISMA-compliant systematic review dedicated to VLA models, offering a structured discussion of robotic policies, VLA architectures, and inference optimization methods. The review also presents descriptive analyses of the included studies and a glossary defining the terminology commonly used in VLA and generalist robotic policy research. The findings reveal substantial diversity among VLA models in terms of their supported modalities, robotic embodiments, training strategies, and architectural designs. Despite the rapid growth of VLA research, several important areas remain underexplored, including the execution of complex, long-horizon tasks, effective integration of speech, and deployment on low-cost hardware, while ensuring robust, safe, and secure operation.
Umair Cheema, Y. Badr, T. Le et al.· Robotics· 0 citations
Open-vocabulary grasping on a quadruped manipulator requires more than recognizing the target object. The robot must also select a grasp pose that is both consistent with the task semantics and reliable to execute under body motion and viewpoint changes. In this paper, we present VLEG, an embodied vision-language grasping framework for quadruped manipulators that explicitly incorporates body motion into grasp decision making. Our method guides the robot to continuously adjust its body pose during approach and optimize local observations before grasping, thereby improving perception quality. For grasp decision making, instead of using a coarse single-stage filtering strategy, we design a multi-stage and multi-criteria grasp selection mechanism based on geometric grasp candidates. This mechanism jointly considers physical feasibility and task consistency. We implement the complete system on an onboard Jetson platform and conduct extensive real-world experiments on a quadruped robot equipped with a manipulator, covering tabletop, low-platform, ground-level, and outdoor raised-platform scenes. The results validate the deployability of VLEG in real-world quadruped manipulation scenarios, as well as its robust grasping ability and task-aware decision-making capability across the tested object categories.
Yu-Xing Ji, Fei Meng, Zishang Ji et al.· Journal of Physics, Conferen...· 0 citations
GCA-Bench is proposed, a benchmark featuring challenging grasping with complex action scenarios that involve both scene-level reasoning and semantic constraints and enables the evaluation of recent large foundation models under the same settings.
Hanyi Zhang, Khang Nguyen, Charith Munasinghe et al.· 0 citations
The remarkable progress of Multimodal Large Language Models (MLLMs) has attracted increasing attention to extending them to physical entities like legged robots and robotic arms. This typically requires MLLMs to not only grasp multimodal understanding abilities, but also integrate visual-spatial reasoning and physical interaction capabilities. Nevertheless, existing methods struggle to jointly accommodate these capabilities within a shared multimodal model, due to the substantial differences between vision-language reasoning and low-level physical control. In this paper, we present Visual Embodied Brain-1.5 (VeBrain-1.5), a task-level unified framework that connects multimodal perception and spatial reasoning with robot control through a shared MLLM-compatible decision interface. VeBrain-1.5 reformulates robotic control into common text-based MLLM tasks in the 2D visual space, thereby aligning the supervision format and prediction interface of multimodal understanding, spatial reasoning, and robot control. A modular robotic adapter subsequently translates these high-level textual decisions into embodiment-specific motion commands and executes them in a closed loop. From the data perspective, we further introduce VeBrain-1M, a high-quality instruction dataset encompassing various capabilities of VeBrain-1.5. In VeBrain-1M, we take hundreds of hours to collect, curate and annotate the data, and adopt multimodal chain-of-thought (CoT) to mix the different capabilities into a single conversation. Extensive experiments on 13 multimodal benchmarks, 7 spatial intelligence benchmarks and 4 simulation and real-world robotic benchmarks demonstrate the superior performance of VeBrain-1.5 over existing MLLMs like Qwen2.5-VL. To further eliminate the task ambiguity and enhance the reasoning capability, we adopt an offline RL policy learning with verifiable reward to increase the task success rate. When deployed to two robotic entities including legged robots and robotic arms, VeBrain-1.5 shows strong adaptability, flexibility, and compositional capabilities compared to existing methods.
Ganlin Yang, G. Luo, Ziyang Gong et al.· IEEE Transactions on Pattern...· 0 citations
