Skip to content
Book Open access

Similarity-Driven Retrieve-and-Refine Category-Level 6D Object Pose Estimation via Geometry-Language Retrieval using Bidirectional Feature Fusion

Nov 2026 · Proceedings of the 34th ACM International Conference on Multimedia · 0 citations · 54 references

Abstract

Category-level 6D object pose and size estimation remains challenging due to severe intra-class geometric variation, which makes direct absolute prediction unreliable for unseen instances. In this paper, we reformulate the task as a similarity-driven retrieve-and-refine problem, where pose estimation is grounded in geometry-semantic analogies rather than one-shot regression. To this end, we propose a geometry-language retrieval stage that first retrieves a small set of pose-valid and semantically consistent exemplars, and then performs candidate-conditioned residual refinement for final prediction. Specifically, we introduce a Siamese Geometry Retrieval Network (SGRNet) with a pose-and-shape-aware metric objective to learn retrieval embeddings aligned with downstream pose validity. We further incorporate language descriptors derived from a multimodal foundation model to regularize retrieval under ambiguous geometry and partial observations. Given the retrieved candidates, a Residual Pose Estimation Network (RPENet) predicts pose and size residuals relative to each candidate, converting the original ill-posed global estimation problem into a set of better-conditioned local refinement problems. Across both stages, we design a bidirectional depth-point fusion module to couple view-dependent depth cues with global 3D shape structure. Extensive experiments on REAL275, CAMERA25, and Omni6DPose demonstrate that the proposed formulation achieves strong generalization and state-of-the-art performance, with additional transferability to instance-level estimation on YCB-Video.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.