Query-Driven Evidence Retrieval for Efficient 3-D Question Answering

Abstract

The integration of Large Vision-Language Models (LVLMs) with 3D scene understanding has shown great promise. However, existing 3D Visual Question Answering (3D QA) paradigms face severe bottlenecks. Directly feeding dense, unconstrained multi-view video streams or full point clouds into LLMs incurs prohibitive computational overhead and attention dilution, rendering models highly susceptible to spatial disorientation and visual hallucinations. To address these challenges, we propose Opti3D, a training-free, plug-and-play structured evidence construction framework that retrieves compact query-relevant visual evidence for off-the-shelf Video LLMs. Specifically, Opti3D constructs a global Bird’s-Eye View (BEV) map to preserve macroscopic spatial layout and further organizes local object observations through explicit geometry-aware cross-view deduplication. Instead of treating every sampled frame as an independent visual input, Opti3D groups redundant 2D observations that correspond to the same physical instance and then selects compact, viewpoint-informative local evidence using a multi-dimensional view-quality score. In this way, complex 3D roaming videos are converted into a structured visual evidence set containing a global BEV context and non-redundant local instance views. Extensive experiments on challenging 3D QA benchmarks demonstrate that Opti3D achieves competitive performance among training-free or frozen-backbone Video LLM baselines, while remaining below specialized 3D LLMs trained with task-specific 3D-language supervision. The results show that explicit geometry-aware view deduplication reduces redundant visual inputs and provides more reliable evidence for viewpoint-ambiguous questions without requiring full supervision or task-specific training.