A feed-forward, spatially constrained framework that grounds task-relevant objects and generates position-aware subtasks without an iterative task–scene refinement loop is proposed and confidence-guided verification improves the unweighted scene-average Relaxed Accuracy F1.
Abstract
3D scene graphs connect spatial perception with high-level language reasoning, but unverified false-positive groundings can impair downstream subtask generation. We propose a feed-forward, spatially constrained framework that grounds task-relevant objects and generates position-aware subtasks without an iterative task–scene refinement loop. The framework performs Agglomerative Information Bottleneck clustering within room-specific spatial scopes and augments accumulated-feature similarity with a best-view raw image crop score. A Large Language Model (LLM) evaluates the combined confidence with spatial and geometric evidence to verify object candidates, and subsequent subtasks reference the retained object set. Controlled experiments across three reconstructed indoor scenes, Office, Apartment, and Cubicle, show that confidence-guided verification improves the unweighted scene-average Relaxed Accuracy F1 from 0.364 to 0.574 and pooled best-instance selection accuracy from 50.9% to 86.0% over 57 tasks. A separate open-set grounding evaluation shows higher precision with LLM-based verification, while the Office case study reveals a trade-off between fewer false positives and reduced retrieval coverage. The framework provides an intermediate scene-grounded subtask specification; residual grounding errors and omitted objects remain limitations for downstream use.
SceneBench is introduced, a benchmark of 966 photorealistic 3D scenes reconstructed with Gaussian Splatting and densely annotated with hierarchical semantics spanning scenes, rooms, functional areas, object groups, and individual objects that provides a realistic testbed for developing and evaluating models capable of...
Anubhav Khanal, Prabigya Acharya, Roshni Poudel et al.· 0 citations
AffordGraph is proposed, an efficient pipeline whose primary contribution is affordance-guided relation filtering that brings functional scene graph construction within the latency and power budget of AR agents and comes at a moderate, controllable cost in recall.
Eunjae Shin, Seokyoung Kim, Woontack Woo· 0 citations
Image-based 3D visual grounding is critical for embodied agents, yet existing benchmarks suffer from loose text-observation alignment and neglect temporal ordering. We introduce SeqAlign3DVG, a novel benchmark dedicated to temporally ordered and strictly observation-aligned image-based 3D visual grounding. Unlike prior...
Yi Zhang, Yi Wang, Yue-Ting Wu et al.· 0 citations
Recent advances in 3D reconstruction have progressed from per-scene optimization to feed-forward inference, and semantic scene understanding has followed suit -- yet existing methods remain confined to object-centric perception, neglecting spatial relations between objects. We formulate 3D spatial relation segmentation...
Minsu Kim, Jaesung Choe, Ji-Woo Lee et al.· 0 citations
Robotic agents use 3D scene graphs (3DSG) to perform tasks ranging from scene understanding to scene interaction. Although an extensive body of work addresses scene graph generation, little attention has been paid to optimizing the graph for consumption, which leaves state-of-the-art perception pipelines to work around...
This work proposes a unified autoregressive diffusion-based graph generative model that jointly learns structure and features, constructing complete 3DSGs bottom-up from observed vertical planes across arbitrary hierarchy depths, and proposes an adaptation of the Fused Gromov--Wasserstein distance for principled graph-...
J. A. Millan-Romera, Samuel Cognolato, Holger Voos et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.