Sep 2026· IEEE Transactions on Pattern Analysis and Machine Intelligence· Vol PP· 0 citations
Medicine
TL;DR
SIG is introduced, a structured grid-based schema that explicitly represents object layouts, inter-object relations, and physically grounded priors that demonstrates the generalizability of SIG across diverse scenarios and multiple MLLMs, and systematically examines how data sample selection influences SIG-based ICL.
Abstract
How to encode and evaluate spatial intelligence in foundation models remains an open challenge. Existing approaches often rely on textual proxies and VQA-style evaluation to assess visual-spatial intelligence (VSI), which can obscure geometric structure, encourage linguistic shortcuts, and hinder attribution to genuinely spatial reasoning abilities. To address this limitation, we introduce spatial intelligence grid (SIG), a structured grid-based schema that explicitly represents object layouts, inter-object relations, and physically grounded priors. As a complementary modality to text, SIG offers a faithful and compositional representation of scene structure for foundation-model reasoning. Building on SIG, we further propose SIG-informed evaluation metrics for measuring a model's intrinsic VSI, thereby disentangling spatial competence from language priors. In few-shot in-context learning (ICL) experiments with state-of-the-art multimodal large language models (MLLMs), including GPT- and Gemini-family models, SIG consistently delivers larger, more stable, and more comprehensive improvements across all VSI metrics than VQA-only representations. These results highlight its potential as an effective annotation and training schema for learning VSI. We further demonstrate the generalizability of SIG across diverse scenarios and multiple MLLMs, and systematically examine how data sample selection influences SIG-based ICL through extensive experiments. In addition, we release SIGBench, a benchmark comprising 1.4K driving frames annotated with ground-truth SIG labels and human gaze traces, supporting both grid-based machine VSI tasks and attention-driven, human-like VSI tasks in autonomous-driving settings.
Space Tokens is introduced, a lightweight, architecture-agnostic framework that equips VLMs with explicit continuous spatial representations without requiring additional inference-time modules, and demonstrates that continuous spatial tokens provide an effective, interpretable, and computationally efficient mechanism f...
Hunter Schofield, Mohammed Elmahgiubi, Mohammad Mahdavian et al.· 0 citations
Spatial-OPSD, a label-free self-improvement framework that instead exploits spatial structure naturally available from perception and reconstruction tools, is introduced, achieving the highest average among the open models and the best results on three of five spatial reasoning benchmarks.
Think with Structured Grounding (TwSG), a novel fine-grained image perception framework designed to internalize complex images's tool-use capabilities within the model, is proposed, endowing models with native fine-grained region description and flexible reasoning capabilities.
Chang-Jiang Jiang, Qian-Nian Zhao, Lei Xin et al.· 2 citations
This survey provides a comprehensive and unified overview of recent advances in spatial intelligence for VLMs, summarize core concepts behind spatial reasoning in VLMs, analyze why spatial failures occur, and organize existing solutions into a clear framework spanning prompting-based techniques, model improvements, exp...
Di-Sheng Liu, Tuo Liang, Zhe Hu et al.· Artificial Intelligence Revi...· 9 citations
This work introduces SpatialBlock-15k, a synthetic dataset of 15,000 block-stacking problems covering 3D-to-2D projection, viewpoint transformation, and structural combination and proposes a novel paradigm inspired by human cognitive development: learning foundational spatial skills through structured block-manipulatio...
It is shown that MVRD makes visual representations more geometric while retaining language alignment, and generalizes to 3D scene understanding tasks such as object grounding, dense captioning, and question answering, while approaching feature fusion methods with considerably fewer added parameters and lower latency.
K. T. Nguyen, Hanbo Shim, Jinwoo Kim et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.