Point Localization and Pose Estimation Based on RGB-D and HSN-YOLOv8n-seg Model for Marigold Harvesting

Abstract

Accurate picking-point localization and pose estimation are essential for selective robotic harvesting of marigolds, yet side-view imaging suffers from slender pedicel morphology, frequent occlusions, and weak visual contrast at the corolla–pedicel junction. This study aims to deliver a reliable perception framework that outputs both the 3D picking-point coordinates and the corresponding grasping pose angle for each harvestable flower. To achieve this, we propose an RGB-D based pipeline that integrates depth-assisted foreground extraction, a lightweight instance segmentation model, a dual-strategy corolla–pedicel association module, and geometric pose estimation. To improve pedicel segmentation under weak-boundary and occlusion conditions, an enhanced lightweight model, HSN-YOLOv8n-seg, was developed by incorporating Haar wavelet downsampling to preserve fine-grained texture features, Spatial Group-wise Enhancement to strengthen responses in ambiguous regions, and a lightweight segmentation head to improve mask boundary delineation. The model achieved a Precision of 78.53% and an mAP50 (mean Average Precision at IoU threshold 0.5) of 75.43%, with only 3.17 M parameters and an inference time of 9.42 ms, outperforming representative lightweight segmentation models such as YOLOv9t and YOLOv11n. The framework computes the pedicel centroid as the candidate picking point and pairs it with the corresponding corolla centroid via distance priority matching and cross-line verification; the axis connecting the paired centroids defines the picking pose. Outdoor experiments showed that the largest group-wise mean localization error was 0.0095 m (individual sample maximum: 0.0107 m), and the largest group-wise mean pose angle error was 5.11° (individual sample maximum: 14.14°). Overall, the framework consistently maintains reliable perception under occlusion and indistinct corolla–pedicel boundaries, demonstrating its potential for automated marigold harvesting. Nevertheless, further validation under diverse and complex field backgrounds is necessary before practical deployment.