Two intentionally minimal simulation foundations for education and rapid prototyping are presented, designed to run locally on macOS, Windows, and Linux and deliberately left open to modification rather than developed as finished applications.
Abstract
Large language models (LLMs) and vision-language models (VLMs) are expanding the range of behaviors that can be represented in agent-based simulations, but many contemporary platforms are difficult to study, modify, or run on ordinary computers. We present two intentionally minimal simulation foundations for education and rapid prototyping. SD-AgentFoundry-2D provides a two-dimensional multi-agent environment in which locally hosted LLM agents move, communicate, respond to place occupancy, and encounter spatially localized fire events. SD-AgentFoundry-3D provides a three-dimensional digital-twin environment in which a locally hosted VLM receives first-person images and produces natural-language movement instructions. Both codebases are designed to run locally on macOS, Windows, and Linux and are deliberately left open to modification rather than developed as finished applications. Together, they offer accessible starting points for learning about generative social simulation and for building domain-specific extensions.
Across seven benchmarks and four base VLMs, VLM-in-Sandbox achieves the highest sample-weighted average accuracy among Vanilla VLM, Append-only Sandbox, and the proposed method, and is identified as a central abstraction for sandboxed VLM agents.
He-Xiong Yang, Ming-Rui Chen, Jie Cao et al.· 0 citations
A distributed conversational framework for human-robot collaborative manipulation that integrates local language and vision-language models (VLMs) with a Robot Operating System 2 (ROS 2)-based execution stack, enabling flexible deployment across distributed hardware while maintaining a responsive control loop.
Arash Ghasemzadeh Kakroudi, Roel Pieters· IEEE International Symposium...· 0 citations
We present a method for teaching vision-language models (VLMs) to perform spatial navigation in grid-based environments using structured supervision derived from distance fields. Our approach combines binary space partitioning for maze generation, Dijkstra-based distance field computation, and A* pathfinding to create...
Ilya Olkov, Alex I. Gavrilov· 2026 International Russian A...· 0 citations
We present MintAct, a family of vision-language models that unifies UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use, trained at 2B, 4B, and 8B scales. Through careful design of our environments, data, and training recipes, MintAct models match the performance of per-domain speci...
Ming-Fei Gao, Rui Tian, Hai-Ming Gang et al.· arXiv.org· 0 citations
A simulation-grounded vision-language model (VLM) framework that automatically converts 2D wildfire simulations into labeled video episodes that connects automatic simulation-to-proxy conversion with memory-based VLM reasoning under scarce real-world physical annotations is presented.
Duo-Wen Chen, Yu-Chen Sun, Zhi-Qi Li et al.· 0 citations
Robion is designed, the first VLA serving and management system for multi-robot, multi-model requests on multi-GPU edge servers that meets SLOs and enables flexible model placements on multi-GPU servers, and integrates an intelligent traffic controller.
Dionysios Adamopoulos, Nattapol Chanpaisit, Basel Fakhri et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.