KNA-SG, a framework for constructing open-vocabulary 3D scene graphs from RGB sequences with explicit keyframe–node associations, is proposed and Experimental results show that KNA-SG outperforms existing methods on open-vocabulary 3D semantic segmentation and 3D object grounding tasks.
Abstract
3D scene graphs organize objects and their relationships in a scene into structured representations, providing an interpretable and queryable foundation for relational reasoning and object grounding. Existing open-vocabulary 3D scene graph construction methods primarily focus on object-level feature representation and open-ended edge reasoning. However, they often lack explicit and retrievable associations between object nodes and keyframes, making it difficult to recall relevant visual evidence for target disambiguation and relationship verification in complex queries. Moreover, pre-constructed edges are inherently limited in their ability to cover the diverse linguistic expressions encountered in downstream tasks. To address these limitations, we propose KNA-SG, a framework for constructing open-vocabulary 3D scene graphs from RGB sequences with explicit keyframe–node associations. Built upon instance-grounded 3D reconstruction, KNA-SG represents each object instance as a graph node and uses a unique instance identifier to associate the node with the keyframes in which the instance is observed. The ID-annotated keyframes guide MLLM-based open-vocabulary semantic parsing, enabling semantic attributes to be assigned to these graph nodes. This design transforms keyframes into retrievable visual evidence for target disambiguation and relationship verification during query reasoning. Verified relationships are further written back into the scene graph as reusable relational memory to support subsequent queries. To ensure the effectiveness of selected keyframes, we design a two-stage keyframe selection strategy that combines visual quality assessment with semantic redundancy removal, preserving a set of clear keyframes that provide comprehensive scene coverage. Experimental results show that KNA-SG outperforms existing methods on open-vocabulary 3D semantic segmentation and 3D object grounding tasks.
Open-vocabulary 3D scene understanding aims to segment 3D scenes beyond predefined categories by transferring semantic knowledge from vision-language models. Existing methods have advanced this task by lifting language-aligned 2D features into 3D, yet they often rely on context-independent semantic representations, leaving object relationships underexplored for contextual refinement. We propose RelGraphOV, a relationship-aware framework that uses 3D scene graphs to enhance open-vocabulary 3D understanding. Our method constructs relational scene graphs from multi-view observations by leveraging vision-language reasoning to infer object relationships and prune geometrically implausible connections, without manual relationship annotations. To aggregate relational context while avoiding feature interference, we introduce an Adaptive Gated Dual-Stream Contextual GAT that separates dense geometric features and semantic CLIP embeddings, performs edge-guided message passing, and adaptively fuses complementary semantics. A hierarchical contrastive objective further promotes instance-level consistency and category-level discrimination. Experiments on ScanNetV2, ScanNet200, ScanNet$++$, and Replica demonstrate strong performance and generalization ability. Project Page: https://cxavireh.github.io/relgraphov-projectpage
Xianhao Chen, Jiarui Hu, Yuanbo Yang et al.· 0 citations
OVIP-SG is presented, a unified framework for instance-preserving semantic mapping, functional scene partitioning, and language-guided small, fine-grained object retrieval that outperforms ConceptGraphs under a unified evaluation protocol on Replica.
Tianjing Hao, Haiyu Lan, Ang Li et al.· 0 citations
Text-to-3D Scene Retrieval (T3SR) aims to retrieve 3D scenes that match users' linguistic queries, enabling intuitive access to 3D scene repositories. Existing approaches rely on joint embedding learning with large amounts of paired text–scene data, which is expensive to collect and often fails to generalize under open-vocabulary queries and diverse scene distributions. In this paper, we propose Struct-Align, a foundation-model-driven framework for zero-shot T3SR that eliminates the need for paired training data. Our key insight is to reformulate T3SR as a single-modality structural alignment problem by converting both 3D scenes and textual queries into a shared, schema-aligned textual representation compatible with pretrained text embedding models. To reliably derive such representations from complex 3D environments, we introduce a role-decomposed scene structuring pipeline that mitigates generative instability and produces semantically consistent scene depictions. To address the inherent semantic asymmetry between query and scene representations, we further propose a locality-aware structural matching strategy that explicitly localizes query intent and performs instance- and relation-level alignment within query-relevant substructures. Extensive experiments on multiple benchmarks demonstrate that Struct-Align outperforms both training-based and zero-shot baselines while exhibiting strong robustness to domain shift.
Xiong Li, Yikang Yan, Zhenyu Wen et al.· Annual International ACM SIG...· 0 citations
This work proposes GroupForward, an instance-grouped feed-forward Gaussian splatting model that reconstructs geometry, appearance, instance structure, and semantics from sparse, unposed, and uncalibrated multi-view images and proposes a Referential Scene Reasoning Framework (RSRF) for complex 3D referring segmentation.
Qijian Tian, Zimeng Wu, Xuhong Wang et al.· 0 citations
A modular mapping architecture is demonstrated that establishes 3D Semantic Scene Graphs (3DSSGs) as its foundational back-end, enabling the dense representation of extensive environments containing thousands of unique object instances and supporting open-vocabulary queries via CLIP features without requiring any additional post-processing steps.
F. Igelbrink, Lennart Niecksch, Martin G. ̈unther et al.· 0 citations