Jul 2026· 2026 23rd International Conference on Ubiquitous Robots (UR)· pp. 199-205· 1 citation· 16 references
Abstract
Reliable manipulation of small, densely stacked objects remains a significant challenge due to severe occlusions and geometric ambiguities. When observed from a single RGB-D viewpoint, adjacent surfaces of featureless cuboids like Jenga blocks often merge in depth measurements, hindering accurate instance separation and pose estimation. This paper presents a unified perception and manipulation framework designed for the robotic rearrangement of randomly stacked Jenga blocks using a single Intel RealSense D435 sensor. Our approach integrates a perception pipeline based on heightmaps, which combines color extraction with geometric reasoning to robustly segment individual blocks and estimate poses directly compatible with grasp planning. To overcome the inherent limitations of sensing from a single view in dense clutter, we propose an iterative strategy of scanning, evaluating, and regrasping. When no immediate grasp is feasible, the system performs controlled actions of grasping and releasing to induce local reconfiguration, transforming cluttered states into graspable arrangements. Experimental results, conducted under conditions equivalent to the actual competition, demonstrate a 99.02% task success rate. The proposed framework ensures stable pick-and-place operations that inherently consider manipulation constraints, proving that robust task execution is achievable using a single RGB-D sensor.
Deploying robots in everyday human environments requires perception systems that are both robust and adaptable to diverse, dynamic conditions. In this work, we present a modular perception pipeline for household manipulation tasks, with a focus on dishware handling in kitchen environments. The pipeline integrates open-vocabulary object detection, multi-view segmentation, instance-aware 3D reconstruction, and a 2D-3D feature fusion strategy for 6D pose estimation and grasp planning. Its modular design enables systematic substitution of multiple visual and geometric foundation models, allowing us to identify the best-performing configuration through extensive evaluation on a custom kitchen dataset. The best-performing configuration (LLMDet + SAMv2 + DINOv2 + GeoTransformer) achieves an ADI of 89.12% on the 20-scene kitchen benchmark with cluttered and occluded conditions. Furthermore, real-world demonstrations confirm that the best configuration can be deployed on physical robots without environment-specific retraining, successfully executing tasks such as sink-to-dishwasher transfer and cup stacking. It validates the adaptability and scalability of the pipeline and highlights its potential as a practical framework for household robotic systems. Our code and supplementary materials are available at https://raivlab.github.io/FM_kitchen/.
A novel 7-DoF grasping pose generation framework that integrates sparse attention and null convolution is introduced, which enhances the model’s ability to capture fine-grained features from point clouds, significantly improving the accuracy of parallel gripping pose estimation.
Hui Zhang, Yue Wang, Kang An et al.· Signal, Image and Video Proc...· 0 citations
A robust, hierarchical 3D pose estimation framework designed to bridge the gap through a coarse-to-fine registration strategy, which completes the perception loop in 0.26 s, facilitating real-time feasibility and systematic bin-picking trials substantiate the operational reliability of the framework.
Juan Zhao, Shuo Yang, Quanguo Fu· Unmanned Systems· 0 citations
Antipodal grasping from single-view red-green-blue and depth (RGB-D) images is challenged by occlusion and partial observability, making purely analytical inference ill-posed. We present the Grasping Framework with Learning-Based Perception and Analytical Modeling (GFLA), which fuses learning-based perception with analytical modeling. GFLA projects antipodal contacts to the image plane, samples grasp candidates via inverse projection, and ranks them with a force-closure metric. To compensate for the information loss inherent in single-view observations, we introduce two grasping hypothesis-guided modules: 1) a contact projection detection network that localizes graspable regions and predicts antipodal projections on visible surfaces, and 2) a 3-D U-Net-based scene completion network that completes geometry and provides explicit collision cues. On GraspNet-1Billion, GFLA achieves its largest improvement on the novel object set (average precision (AP) 35.88%, an improvement of 7.59%), demonstrating superior generalization to previously unseen object categories while also attaining a competitive overall AP of 57.84% (an improvement of 1.33%). Real-robot experiments in cluttered environments, without domain adaptation or fine-tuning, achieve grasp success rates of 95.42% for single-object scenes and 90.12% for multiobject scenes, demonstrating strong practical robustness.
Xiao Ning, Jianzhong Yang, Si Huang et al.· IEEE Transactions on robotic...· 0 citations
To address the core issues faced by edge-side 3D scene perception of mobile robots, such as the difficulty of balancing accuracy and real-time performance, insufficient multi-task fusion, and deviations in depth geometric consistency, this paper proposes a depth-guided lightweight multi-task 3D scene understanding method. This method is based on a lightweight U-Net architecture, uses RGB and depth four-channel dual- modal input, and constructs a network structure with a shared encoder and independent decoders, allowing a single inference to simultaneously accomplish the three core tasks of semantic segmentation, depth completion, and obstacle detection. It also Introduce depth consistency error (DCE) to construct a weighted joint loss function, strengthening scene geometry structure learning; through a dual lightweight strategy of structured pruning and INT8 quantization, precisely adapt to NVIDIA Jetson Xavier NX edge hardware. Experimental results on the NYU Depth V2 and KITTI datasets show that this method achieves a semantic segmentation mIoU ≥ 60%, depth completion RMSE ≤ 1.0m, obstacle detection F1-Score ≥ 85%, and edge- end inference speed ≥ 15 FPS, effectively balancing perception accuracy and real-time performance, providing a highly practical lightweight solution for autonomous perception in mobile robots.