Skip to content
Preprint

HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesis

Sep 2026 · 2 citations · 36 references
Computer Science

TL;DR

HARMONY is a hierarchical chain-of-thought framework that leverages both agentic reasoning and visual geometry foundation that can produce 3D scenes that are semantically consistent and perceptually aligned with the reference image, extending single-image compositional reconstruction to complex indoor scene images.

Abstract

Compositional 3D scene reconstruction has recently been explored from two directions: agentic reasoning that provides semantic understanding of spatial relationships but lacks precise alignment with input images; and visual geometry foundation models that predict dense point maps from input images but the reconstruction quality is limited. Therefore, recovering a complete 3D scene from a single monocular image with accurate inter-object relationships and high-fidelity reconstruction quality remains challenging. In this paper, we present HARMONY, a hierarchical chain-of-thought framework that leverages both agentic reasoning and visual geometry foundation. Given an image of an indoor scene, starting from an empty 3D floorplan, HARMONY first calibrates the camera against the reference image to establish a semantically-grounded spatial frame, then uses agentic VLM reasoning to recover the 3D room layout and an initial placement order. It then places the objects in a hierarchical order, from wall-mounted elements, free-standing furniture, to dependent decorations on top of furniture. We also use depth-first traversal for furniture so each placement conditions on previously resolved structure and a reflective feedback loop to avoid error accumulation. After each object placement by VLM, we use the point cloud estimations to perform geometry-based refinement so that the rendered image aligns better with the input. HARMONY can produce 3D scenes that are semantically consistent and perceptually aligned with the reference image, extending single-image compositional reconstruction to complex indoor scene images. Experiments on synthetic and real-world images demonstrate that HARMONY outperforms the evaluated reconstruction baselines, while qualitative comparisons with GPT-6 Astra suggest more faithful object arrangements and better preservation of scene details.

View source

Similar papers

Preprint Sep 2026

SPOON: Towards Coherent Compositional 3D Scene Generation from Uncalibrated Multi-view Images

SPOON is introduced, a framework that reformulates multi-view compositional 3D generation as scene-level, geometry-grounded pose reasoning and progressively organizes object poses and camera configurations into a coherent scene-level spatial arrangement.

Gui-Biao Liao, M. Xiang, Heng Li et al. · 0 citations
#artificial intelligence Preprint Sep 2026

SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes

SceneBench is introduced, a benchmark of 966 photorealistic 3D scenes reconstructed with Gaussian Splatting and densely annotated with hierarchical semantics spanning scenes, rooms, functional areas, object groups, and individual objects that provides a realistic testbed for developing and evaluating models capable of...

Anubhav Khanal, Prabigya Acharya, Roshni Poudel et al. · 0 citations
Sep 2026

LEGO-Anything: Coding Agents for 3D Scene Reconstruction

These readouts suggest that program-constructed scenes from current coding agents are a promising but not yet sufficiently precise representation of natural images, suggesting that program-constructed scenes from current coding agents are a promising but not yet sufficiently precise representation of natural images.

Xi-Rui Li, Peng Shi, Ming-Wen Dong et al. · 1 citation
#artificial intelligence Preprint Oct 2026

OmniAct3D: Leveraging Foundation Geometry and Evidence-Grounded Reasoning for Panoramic 3D Detection

Accurate 3D detection is essential for mobile embodied agents, while Vision Foundation Models (VFMs) offer transferable visual and geometric priors. Yet existing VFM-based 3D detectors rely on narrow-view monocular images or discrete perspective views, limiting coherent surround perception; equirectangular projection (...

Run-Tong Wu, Fei Teng, Di Wen et al. · 0 citations
Sep 2026

Fysiverse-3D-Vision Technical Report: Generating Executable 3D Worlds from Images through Unified Spatial Reasoning

Fysiverse-3D-Vision is proposed, a unified vision-language-geometry framework for generative 3D scene reconstruction and executable asset construction from a single image that establishes a shared representation where spatial reasoning and geometric reconstruction mutually enhance each other, allowing object layouts to...

Ding-Kang Yang, Yi-Zhou Liu, Wen-Dong Cheng et al. · 0 citations
#machine learning Preprint Sep 2026

CODA: Depth-Aligned Scene Completion and Object Decomposition from a Single RGB-D Image

Robots operating safely in cluttered everyday environments often need to infer scene geometry from partial observations. Methods that detect objects in 2D and reconstruct them independently struggle in such scenes: a missed object is never reconstructed, a merged detection can fuse two objects, and separately reconstru...

Dongwon Son, Junhyek Han, Yoon-Je Cho et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.