This work proposes ThinkBLOX, a VLM-based progressive reasoning framework that iteratively designs and refines 3D scenes and introduces Tier-Decoupled GDPO, a reinforcement learning scheme that organizes heterogeneous rewards into distinct tiers, stabilizing policy optimization across physical validity, semantic plausibility, and reasoning-action consistency.
Abstract
While traditional graphics methods often synthesize 3D indoor scenes autoregressively or hierarchically, recent vision-language model (VLM)-based generators predominantly adopt a one-shot paradigm where the full layout is planned at once. This one-shot approach often requires global re-optimization or complete reconstruction during interactive editing (e.g., inserting or moving objects) and can lead to physically or semantically poorly organized arrangements. To address these challenges, we propose ThinkBLOX, a VLM-based progressive reasoning framework that iteratively designs and refines 3D scenes. ThinkBLOX treats layout generation as a state-conditioned, step-by-step reasoningand-action process. To power this, we construct the ThinkBLOX-Data-200K dataset, containing 224,757 procedural placement pairs annotated with multi-view scene context, explicit Chain-of-Thought (CoT) rationales, and structured JSON layouts. Through supervised fine-tuning (SFT) on this dataset, the VLM learns to bridge the reasoning-action gap under incremental updates. Furthermore, recognizing that scene synthesis is inherently a multisolution task where SFT suffers from reward conflict, we introduce Tier-Decoupled GDPO. This reinforcement learning scheme organizes heterogeneous rewards into distinct tiers, stabilizing policy optimization across physical validity, semantic plausibility, and reasoning-action consistency. Extensive experiments show that ThinkBLOX significantly outperforms recent one-shot and iterative baselines in physical plausibility, semantic alignment, and interactive editability. Additionally, we show that it supports diverse applications, including both global and local generation and rearrangement of 3D scenes.
SceneBench is introduced, a benchmark of 966 photorealistic 3D scenes reconstructed with Gaussian Splatting and densely annotated with hierarchical semantics spanning scenes, rooms, functional areas, object groups, and individual objects that provides a realistic testbed for developing and evaluating models capable of...
Anubhav Khanal, Prabigya Acharya, Roshni Poudel et al.· 0 citations
DesignAgent3D is presented, an interactive multimodal agentic framework that reformulates 3D scene editing as a designer-like Plan-Perceive-Act paradigm, delivering superior semantic intent alignment, impeccable spatial localization accuracy, and high-fidelity multi-view consistency.
Xiu-Jin Liu, Tian-Yu Yang, Yilun Zhao et al.· 0 citations
By combining HRAP with RMR, ScenePilot offers an efficient alternative to one-shot generation and heavy full-scene optimization, improving physical plausibility, functional coherence, and controllability while preserving diversity.
Jiawei Zhang, Hong-Song Wang, Pan Zhou· 0 citations
Function-Room Generation is introduced, a new indoor 3D scene generation setting that creates rooms supporting explicit functional goals rather than merely visually plausible layouts, and ScenePRM, an execution-grounded process reward framework that improves the expert through reinforcement learning with functional, ge...
Hao Feng, Zhi Zuo, Ming-Jian Liang et al.· 1 citation
ViewMind3D is presented, a fully training-free and modular framework for 3D spatial reasoning over multi-view observations of a scene without requiring complete 3D reconstruction, which improves performance on spatially grounded question types, such as ``What''questions in SQA3D, while maintaining strong overall accura...
Ping-Kun Chiang, Kun-Ru Wu, Po-han Li et al.· arXiv.org· 0 citations
CausalSplat is a framework that integrates vision-language models with 3D scene graphs to disentangle explicit structural perception from implicit logical inference and achieves state of the art performance on reasoning benchmarks while showing strong generalizability on standard referring and open vocabulary 3D segmen...
Jiayu Ding, Meilu Song, Yun Chen et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.