Skip to content
Preprint

A Simulation-Grounded Agentic VLM Framework for Wildfire Monitoring and Reporting

Oct 2026 · 0 citations · 61 references
Computer Science

TL;DR

A simulation-grounded vision-language model (VLM) framework that automatically converts 2D wildfire simulations into labeled video episodes that connects automatic simulation-to-proxy conversion with memory-based VLM reasoning under scarce real-world physical annotations is presented.

Abstract

Effective wildfire monitoring requires relating visual evidence to physical fire dynamics, yet real videos with synchronized physical annotations are scarce and high-fidelity 3D simulation is costly. We present a simulation-grounded vision-language model (VLM) framework that automatically converts 2D wildfire simulations into labeled video episodes. A fixed Blender mapping produces low-detail 3D proxies aligned with simulator terrain, fuel layout, fire activity, and wind cues; controllable video generation supplies richer appearance. The proxies are intermediate representations rather than finely rendered final scenes. Generated videos and simulator labels form reusable multimodal memory for a training-free multi-agent VLM system that retrieves reference episodes, reconciles visual and memory-based predictions, and produces structured wildfire reports. On held-out generated episodes, video memory achieves 51.5% exact four-tag accuracy, compared with 22.6% for direct VLM querying and 16-17% for text-only memory; the complete system achieves 77.3% accuracy on six simulator-derived report fields. Component ablations, cross-generator tests, and three real-UAV evaluations assess retrieval, reporting, generator changes, and observable monitoring tasks. The framework connects automatic simulation-to-proxy conversion with memory-based VLM reasoning under scarce real-world physical annotations.

View source

Similar papers

Preprint Oct 2026

Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos

Building interactive simulators from real-world observations is a promising way to scale embodied data, but current pipelines still rely heavily on manual environment construction and calibration. We study whether frontier foundation models and coding agents can automate this process end to end. We formulate \emph{auto...

Jin-Zhou Tang, Z. Zhang, Jing Yang et al. · 2 citations
Preprint Nov 2026

FlowBench: Counterfactual Imagination Training for Spatial Intelligence

Humans imagine scene changes under hypothetical motions, an ability central to spatial reasoning and robotic planning. However, this capability remains underexplored in vision-language models and world models. We formalize this capability as Counterfactual Spatial Imagination (CSI) and introduce FlowBench, a resource f...

Yi-Fan Liu, Bai-Han Zhang, Khai Loong Aw et al. · 0 citations

Video2STL: Grounding VLM-Generated Temporal Specifications for Robot Learning

Video2STL is presented, a framework that converts observation-only videos into parametric Signal Temporal Logic (STL) specifications and uses the resulting formal representation for robot learning, and supports cross-embodiment transfer from human or animal videos to robot control.

Merve Atasever, Keyan Azbijari, Cagan Bakirci et al. · 0 citations
Preprint Oct 2026

DuoCook: An Overcooked-Inspired Benchmark for Visual Planning and Coordination

Coordinating two players in a cooking task requires a vision-language agent to track changing scene states, sequence recipe steps, and allocate shared resources under a time budget. We introduce an Overcooked-inspired benchmark for evaluating these capabilities in Unreal Engine environments constructed using coding age...

Zi-Yi Jiao, Heng Zhang, Drishti Regmi et al. · 0 citations
Preprint Oct 2026

X-World 1.6: Building Controllable Virtual Cities with World Models

Deploying generative world models for end-to-end closed-loop evaluation and policy learning requires precise scenario control, consistent observations, and sustained generation quality during extended interactions. This report presents \emph{X-World~1.6}, advancing X-World~1.0 toward a more reliable and versatile drivi...

XPeng Boyang Wang, Bo-Yi Sun, Chao-Da Zheng et al. · 0 citations
Preprint Oct 2026

Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation

Generating useful robot-training data requires more than visually plausiblescenes: objects must support interaction, placements must remain physicallyvalid, and tasks must admit repeatable execution. We present\textbf{Awomo-SimDataEngine}, an agentic system that connects asset and scenegeneration to robot demonstration...

Awomo-PhysicalRSI Team, Dan-Jiao Ma, En-Hui Ma et al. · 2 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.