ContactWorld: What Representations Matter for Vision-Tactile World Models in Contact-Rich Manipulation

Zhi-Yuan Zhang Po-Kuang Zhou Kai-Di Zhang A. Desai T. Amosa Davood Soleymanzadeh Jiu-Zhou Lei Ming-Hui Zheng Yu She Department

· 0 citations · 58 references

TL;DR

This systematic study of vision-tactile representations across 12 contact-rich manipulation tasks finds that representations preserving spatial structure and temporal continuity consistently support more accurate prediction and stronger planning performance.

Semantic Scholar

View source

Similar papers

#artificial intelligence Preprint Sep 2026

DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation

Dexterous manipulation depends on contact dynamics that are often only partially observable from vision. Recent World-Action Models (WAMs) couple predictive video world modeling with action generation, but remain largely vision-centric and therefore cannot directly model these contact dynamics. We present DexTacWAM, a...

Hao-Ran Yuan, Ze-Kai Wang, Boning Shao et al. · 0 citations

Preprint Aug 2026

VISTA: Visually Inferred Spatial ConTact Attention for Contact-Rich Manipulation

Contact-rich manipulation requires precise interaction feedback. While vision-centric imitation learning is prevalent, external visual observations provide indirect and ambiguous cues about contact states, particularly under occlusion or subtle object--gripper interactions; dedicated tactile or force sensors can provid...

Jiaying Chen, Wen-Long Dong, Yan Huang et al. · 0 citations

#machine learning Preprint Sep 2026

Dexterous Tactile World Model

World models for manipulation are typically trained from video, yet the events that determine how manipulation unfolds, such as making and releasing contact, are difficult to observe visually and are often easier to sense through touch. We present the Dexterous Tactile World Model (DTWM), a video world model for future...

Zi-Yao Zeng, Xiatao Sun, Hao Wang et al. · 0 citations

Preprint Sep 2026

TACTIC: Understanding Tactile Encoders and Conditioning for Contact-rich Robot Manipulation Policies

Tactile information is essential for contact-rich manipulation tasks in robotics. Vision-based tactile sensors make it particularly easy to design end-to-end manipulation policies with tactile sensing, as they enable the use of existing encoders from computer vision. However, this has led to a huge variety of architect...

Seongjin Bien, Débora Oliveira Makowski, Carlo Kneissl et al. · 0 citations

Preprint Sep 2026

HapticWAM: Distilling Imagined Touch into a World-Action Model without Inference-Time Tactile Sensing

Contact-rich manipulation requires estimating forces, slip and contact geometry that can remain ambiguous in scene images. Optical tactile sensors provide both visual observations of the contact surface and mechanical measurements, yet learning from these signals raises two challenges: representing contact beyond appea...

M. Sannikov, I. Mikhalchuk, Konstantin Gubernatorov et al. · 0 citations

Preprint Sep 2026

When Does Touch Matter? Charting the Vision-Interaction Gap in Cluttered Dexterous Grasping

Dexterous grasping in clutter poses a basic sensing question: when do tactile measurements and external wrench estimates improve on visual geometry? Occlusion and contact can obscure grasp quality, motivating a controlled evaluation of these interaction signals. We present a controlled real-world study over five tablet...

Hao-Lai Jiang, Luis Dominguez, Daniel Seita · 0 citations