Aug 2026· Machine Vision and Applications· Vol 37· 0 citations· 24 references
TL;DR
A novel neural network architecture that accepts natural language-labeled keypoint graphs as prompts and predicts the image coordinates of graph nodes and achieves strong performance on the public MP-100 benchmark while offering greater flexibility in representing and localizing complex objects.
Abstract
As industries increasingly move towards automation, the ability to visually localize objects and accurately estimate their pose, including rigid and non-rigid transformations, becomes critical. Conventional pose estimation methods are constrained to specific object categories, whereas category-agnostic approaches remain limited in accuracy and in their ability to handle composite objects with repeated and non-unique components. In this paper, we present a general framework for object localization based on keypoint graphs, where nodes represent keypoints and edges encode pairwise relations. Both nodes and edges are annotated with natural language descriptions. We introduce a novel neural network architecture that accepts natural language-labeled keypoint graphs as prompts and predicts the image coordinates of graph nodes. Furthermore, we make several industrial datasets publicly available and demonstrate that our method substantially outperforms existing methods. Additionally, we achieve strong performance on the public MP-100 benchmark while offering greater flexibility in representing and localizing complex objects.
This framework combines simultaneous localization and mapping (SLAM), visual-language feature extraction, incremental semantic and instance label fusion, and spatial graph construction to enable a construction robot navigation framework that supports open-vocabulary language queries.
Charles M. Raines, I. Fernandez, Mandy Sun et al.· Journal of computing in civi...· 0 citations
Object pose estimation is a fundamental problem in 3D vision. Although recent state-of-the-art approaches achieve strong performance, they often overfit to existing benchmarks and exhibit limited generalization to novel categories and unseen scenes. We propose UniPose9D, a category-agnostic foundation model for 9D object pose estimation: given an instance mask/ROI and either an RGB-D observation or an RGB image with predicted depth, the model estimates rotation, translation, and metric size without category labels, CAD models, mean-shape priors, or reference views. Specifically, UniPose9D samples point pairs from the observed object geometry and uses DINOv2 and PointNet features to predict NOCS coordinates for each pair. To improve accuracy, we introduce a point-pair-based RANSAC N-hop Kabsch--Umeyama algorithm with an adaptive threshold. We further employ flow matching to address symmetric ambiguities and construct a large-scale training set by curating and aligning pose annotations from existing public datasets. Experiments across six datasets show that a single unified model can match or surpass specialist methods while generalizing to unseen objects and in-the-wild scenarios. Our code and model are available on https://github.com/qq456cvb/UniPose9D.
Yang You, Yiwen Du, Col Harrison et al.· 0 citations
GSP++ is presented, a graph-based inference framework that exploits the manifold structure of test-time point clouds to refine VLM scores without additional training, and introduces a self-training strategy that selects high-confidence positive and negative samples and assigns them calibrated pseudo scores to further stabilize propagation.
Tiankai Chen, Yushu Li, Adam Goodge et al.· IEEE Transactions on Pattern...· 0 citations
Lang2Graph is presented, an experimental framework for indoor topological graph inference from natural-language navigational instructions that isolates four governing factors: instruction structure, metadata clarity, prompting strategy, and model size and reasoning capability, and Reasoning-aligned open-source models of moderate scale (14B parameters) outperform larger proprietary models on the most challenging instruction categories.
Moamin Ibrahim, Yaqoob Ansari, Khaled A. Harras et al.· International Conference on...· 0 citations
3D semantic segmentation has achieved remarkable advances with powerful architectures and large-scale datasets. However, most existing approaches perform under closed-set assumptions, limiting their applicability in dynamic real-world environments where novel categories frequently emerge. This motivates the task of open-world semantic segmentation (OWSS), in which a model must not only identify unknown objects at inference time but also incorporate them with previously learned classes once annotated. To address this challenge, we propose PointPDF V2, a unified framework that integrates open-set recognition (OSS) and incremental learning (IL) into a cohesive pipeline. Our design consists of three components: a multistage pseudo-labeling (MPL) module that generates pseudo-labels for unknown classes by leveraging confidence and geometric information; a distribution-aware confidence estimation (DCE) module that models class-wise semantic distributions to separate known and unknown categories; and a weighted knowledge distillation (WKD) module that balances old-class retention with novel-class adaptation in IL. In addition to the standard OWSS setting, we introduce a more challenging continual OWSS (COWSS) protocol in 3D, where models must simultaneously preserve the known-class performance, acquire new knowledge, and still identify the remaining unknowns across sequential updates. Extensive experiments on both indoor and outdoor benchmarks show that PointPDF V2 consistently outperforms state-of-the-art baselines in OSS, IL, and C-OWSS. We shall release our code and models upon publication of this work.
Jinfeng Xu, Xianzhi Li, Yixue Hao et al.· IEEE Transactions on Pattern...· 0 citations
One common approach to pose estimation involves predicting object keypoints in an image, followed by using Perspective-n-Point algorithms to compute the object's rotation and translation relative to the camera. While rotations preserve object shapes, this property is often neglected in keypoint-based pose estimation methods, where keypoints are typically predicted independently from each other. As imprecise keypoint predictions negatively affects pose estimation accuracy, it also limits its reliability in downstream tasks. In this work, we explore whether such inaccurate pose estimates can be identified by simply examining spatial locations between 2D keypoints. We propose a set of hand-crafted geometric features that capture the self-consistency of keypoint predictions, including pairwise distances, reprojection consistency, as well as render and mask consistency. Despite its simplicity, a logistic regression classifier trained on these features reliably detects pose estimation failures, outperforming confidence-based approaches like conformal keypoint predictions that rely solely on keypoint uncertainty.