SAOG: Spatial-Attribute-Aware 3D Object Grounding via Visual Question Answering-Enhanced Reasoning

Abstract

3D object grounding aims to identify the target object referred to by natural language instructions in the presence of multiple distractors. The task is inherently challenging, as it demands joint consideration of spatial relations and intrinsic object attributes to disambiguate the referred target from distractors. Conventional supervised learning methods often exhibit limited generalization due to the complexity of the problem and the scarcity of annotated data, and offer little interpretability of grounding decisions. Recent zero-shot LLM-based approaches have demonstrated promising results for this task, yet they tend to focus either on tool-based spatial verification or on visual reasoning over images, without jointly accounting for spatial relations and intrinsic attributes. To bridge this gap, we propose SAOG, a zero-shot spatial-attribute-aware 3D object grounding pipeline, in which context-aware visual question answering (VQA) enhances the tool-based reasoning process by synergistically exploiting visual attributes and complementary spatial cues. Experiments on the Nr3D dataset show an accuracy improvement of 7.0 percentage points over state-of-the-art zero-shot methods. We further demonstrate its applicability in robotic navigation, and the code will be made publicly available to facilitate future research.