IMBENCH is introduced, a benchmark designed to evaluate intuitive manipulation as an integrated capability spanning perception, physical reasoning, action generation, and iterative execution, and position IMBENCH as a step toward evaluating and enabling more integrated, adaptive physical intelligence.
Abstract
Humans combine reasoning and motor control to solve complex manipulation tasks under diverse constraints. They build an understanding of the physical world that helps them convert reasoning into actions and quickly adapt to new scenes, tasks, and rules. We refer to this capability as intuitive manipulation. Existing benchmarks fail to capture this integration: they evaluate physical reasoning in isolation from execution, or measure policy performance without requiring explicit reasoning. We introduce IMBENCH, a benchmark designed to evaluate intuitive manipulation as an integrated capability spanning perception, physical reasoning, action generation, and iterative execution. Our tasks require models to infer task-relevant physical structure and generate feasible action sequences under explicit constraints, including contact-rich manipulation, tool use, and multi-stage dependencies. We introduce a benchmark of 35 tasks, 14K filtered trajectories, and scalable tools for generating diverse scenarios. Experiments reveal a consistent gap: vision language models show partial physical reasoning ability but fail to produce executable plans, while state-of-the-art vision-language-action models struggle to satisfy task constraints and generalize across scenarios. These results identify intuitive manipulation as a missing axis in current foundation models and generalist robot policies, and position IMBENCH as a step toward evaluating and enabling more integrated, adaptive physical intelligence.
Vision-Language-Action (VLA) models have demonstrated strong performance in robot manipulation by leveraging pre-trained vision-language models to map observations directly to actions. However, existing approaches reason primarily at the visual or semantic level, lacking explicit understanding of the physical interactions that fundamentally govern manipulation tasks. In this paper, we propose Physics Reasoning VLA, a method that enables VLA models to explicitly reason about physical interactions prior to acting, grounded in two fundamental quantities: contact points, which specify where the target object interacts with the robot or surrounding environment, and contact forces, which describe the magnitude and direction of force applied at those locations. Rather than directly mapping observations to actions, our model first predicts contact points and forces at the pixel level via learnable physics queries, then incorporates the resulting physics-aware features alongside visual and language inputs to generate actions. To prevent physics reasoning from disrupting pre-trained visual and linguistic representations, we further introduce a hybrid attention mechanism that applies full attention over image, language, and proprioceptive tokens, while applying causal attention over physics query and action tokens. We evaluate our method on the RoboCasa simulation benchmark, demonstrating that physics reasoning consistently improves performance over the vanilla π0 baseline, with an average success rate improvement from 36.2% to 43.6%.
Kangmin Kim, Geonhyup Lee, Sangbeom Lee et al.· 2026 23rd International Conf...· 0 citations
Experimental results in various task scenarios show that the proposed framework consistently improves overall task success rates compared with unimodal settings with different LLMs and achieves a higher success rate compared to using only visual or force data.
Existing robot datasets remain expensive to curate, embodiment-specific, and insufficiently annotated with the fine-grained structure required for generalizable reasoning, execution, or long-horizon environment dynamics simulation. Building on our prior work, RoboInter1.0, we present RoboInter1.5, an extended and holistic suite of intermediate representations for both robotic manipulation and embodied world modeling. RoboInter1.5 provides a unified resource of data, benchmarks, and models centered on dense manipulation-oriented intermediate representations. Specifically, RoboInter-Data contains over 230k manipulation episodes across 571 scenes with dense per-frame annotations covering more than ten types of intermediate representations, including subtasks, primitive skills, object and gripper grounding, segmentation, affordance, grasp poses, contact points, motion traces, etc. Built upon these annotations, RoboInter-VQA introduces spatial and temporal embodied VQA tasks to benchmark and improve the intermediate-representation reasoning capabilities of our RoboInter-VLM. RoboInter-VLA further studies how such representations benefit action execution through implicit, explicit, and modular plan-then-execute paradigms. To better model the physical world, we further introduce RoboInter-World, which leverages intermediate representations as structured conditioning signals for controllable prediction of future world states. Extensive evaluations demonstrate that RoboInter1.5 provides a unified spatiotemporal scaffolding for intermediate representations. Rather than treating intermediate representations merely as interpretable signals, RoboInter1.5 conceptualizes them as a bidirectional interface that both regularizes low-level action spaces and constrains the latent rollouts of open-world physical simulators.
Ziqin Wang, Hao Li, Weijun Wang et al.· 0 citations
Robotic systems are deeply embedded in both industry and everyday life, where they are expected to act with speed, precision, and reliability. Classical control and planning methods have long delivered strong guarantees, but often at the cost of computational efficiency and adaptability. More recently, learning-based approaches have shown promise in overcoming these limitations, enabling agents to leverage experience to accelerate decision-making and address previously intractable problems. In this work, we bridge these two approaches through a neuro-symbolic perspective on nonlinear motion planning. Inspired by the Thinking Fast and Slow paradigm, we introduce a dual-process architecture that combines the strengths of robust reasoning and learning. Our framework integrates state-of-the-art symbolic solvers as a ``System-2''component with experience-driven ``System-1''modules. A metacognitive controller dynamically orchestrates their interaction, selecting when to rely on fast intuition versus slower, more precise reasoning. By evaluating the framework across diverse nonlinear benchmark environments, we demonstrate that this architecture yields consistent gains in planning efficiency, accuracy, and generalization, while promoting reuse across tasks. The results suggest that tightly coupling learning with structured reasoning offers a scalable path toward more capable and adaptive robotic systems.
Jia-Yi Yan, Francesco Fabiano, Alessandro Abate· 0 citations
This work proposes APIVOT, a VLM-based planner that adaptively interleaves language and visual thoughts for long-horizon planning that outperforms general-purpose VLMs and prior planning frameworks, achieving the largest gains in spatially constrained settings.