Skip to content
Preprint

SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators

Sep 2026 · 1 citation · 45 references
Computer Science

TL;DR

Experiments show that SyncWorld can accurately simulate action outcomes in previously unseen settings, and that its capability of simulating rollouts enables test-time policy improvement without training.

Abstract

World models are increasingly used as policy-in-the-loop imagination environments, where reliable rollouts require fine-grained controllability with respect to low-level robot actions. A key obstacle to scaling such models in robotics is that actions are not a universal language in pixel space: changes in visual environment, camera view, robot placement, or embodiment alter how the same numerical action manifests visually, leading to conflicting supervision under mixed training and brittle generalization at deployment. We introduce SyncWorld, an action-conditioned world model that serves as a zero-shot simulator across unseen environments without any additional training. SyncWorld leverages a visual calibration episode---paired frames and actions that showcase all the controllable degrees of freedom---to specify the setup-specific Action--Visual Mapping in context. Training with visual calibration contexts teaches the model to interpret actions through visual evidence and to leverage interaction history when explicit calibration is unavailable. Experiments show that SyncWorld can accurately simulate action outcomes in previously unseen settings, and that its capability of simulating rollouts enables test-time policy improvement without training.

View source

Similar papers

Sep 2026

ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulation

This work presents ZYT-World, a single architecture that natively generates four fisheye views with field of view>180{\deg} and three pinhole views and a 19M-parameter variational autoencoder decoder decoder, W8A8 quantization, and the inference engine reduce decoding, backbone, and incremental-execution costs, respect...

Bo-Ni Hu, Xiong Wei, Hao-Ming Huang et al. · 1 citation · ⚡1
Sep 2026

WorldLine: Action-Driven Visual Simulation for Robotic Manipulation

WorldLine is introduced, an action-driven visual simulator that decouples transferable dynamics learning from heterogeneous action grounding and makes WorldLine a scalable and efficient visual simulator for policy evaluation and embodied planning.

Shenghe Zheng, Wen-Bo Li, Ji-Yao Zhang et al. · 1 citation
Sep 2026

Self-Adaptive VLA for Robust Robot Deployment

While Vision-Language-Action (VLA) models demonstrate impressive capabilities in robotic manipulation, their memoryless nature renders them brittle to test-time environment shifts, particularly hardware shifts caused by wear or imperfect calibration. Enabling these models to self-adapt during deployment without requiri...

Hong-Xin Zhang, Chun-Tse Lin, T. Wang et al. · 1 citation

Video2STL: Grounding VLM-Generated Temporal Specifications for Robot Learning

Video2STL is presented, a framework that converts observation-only videos into parametric Signal Temporal Logic (STL) specifications and uses the resulting formal representation for robot learning, and supports cross-embodiment transfer from human or animal videos to robot control.

Merve Atasever, Keyan Azbijari, Cagan Bakirci et al. · 0 citations
Preprint Sep 2026

Proprioception-Anchored Cross-Modal Pretraining for Zero-Shot Sim-to-Real Contact-Rich Assembly

PACE (Proprioception-Anchored Cross-Modal Encoder) is presented, which supervises temporal visual and F/T representations by predicting proprioceptive state transitions and is robust to perturbations that substantially degrade pose-based and learned-fusion baselines.

Yu-Han Wang, Yurou Chen, Hong-Ye Jiang et al. · 0 citations
Preprint Sep 2026

R2S-Eval: Robot Evaluation with Real-to-Sim Calibration via Vision-Language Models

Experiments in both simulation and real-world settings demonstrate that R2S-Eval produces reliable and stable policy conclusions, achieves agreement with human preferences, substantially reduces repeated hardware-operation effort, and reveals behavior-quality differences that are not captured by binary success labels.

Yi-Di Wang, Fei-Xiang Ruan, Ruo-Qu Chen et al. · 2 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.