HapticWAM is introduced, a world-action model that combines heterogeneous tactile encoding, structured contact prediction and teacher-student distillation that jointly generates actions and a contact package describing future events and mechanics.
Abstract
Contact-rich manipulation requires estimating forces, slip and contact geometry that can remain ambiguous in scene images. Optical tactile sensors provide both visual observations of the contact surface and mechanical measurements, yet learning from these signals raises two challenges: representing contact beyond appearance and transferring its benefits to a policy that does not require fingertip observations at deployment. We introduce HapticWAM, a world-action model that combines heterogeneous tactile encoding, structured contact prediction and teacher-student distillation. Its teacher encodes gel images together with deformation, shear, distributed forces, resultant wrench and derived contact state into a frozen video backbone. Rather than predicting tactile pixels alone, the model jointly generates actions and a contact package describing future events and mechanics. Anticipatory Contact Coupling uses the previously imagined package to condition attention, preserving a contact-related input when direct tactile observations are unavailable. Haptic-Imagination Distillation transfers both contact futures and action predictions to a student that retains the generative contact head but removes its fingertip input branches. On a real-world setup, across three contact-rich pick-and-place tasks, HapticWAM Student achieves a 77% per-task mean success rate (41 of 50 starts, 82% pooled), reaching 95% on one of the tasks, outperforming the evaluated teacher and baseline configurations.
HapticWorld is presented, an interactive world simulator that predicts joint torque together with observations and renders it back to the operator in real time, closing the haptic loop between a human and a learned world model and demonstrating that HapticWorld can serve as a stand-alone F/T-conditioned policy evaluati...
Shao-Ting Peng, Li-Tian Liang, Yixuan Wang et al.· 0 citations
This systematic study of vision-tactile representations across 12 contact-rich manipulation tasks finds that representations preserving spatial structure and temporal continuity consistently support more accurate prediction and stronger planning performance.
Zhi-Yuan Zhang, Po-Kuang Zhou, Kai-Di Zhang et al.· 1 citation
Contact-rich dexterous manipulation requires policies that translate physical feedback into motion commands while regulating interaction loads across evolving multi-contact interactions. This requires haptic observations of contact state and action supervision showing how commands should adapt. Existing policies often...
Nai-Sheng Ye, Yin-Zhe Zhou, Jun-Kai Zhao et al.· 0 citations
A visuo-tactile WAM that encodes each fingertip independently, aggregates the resulting features through a finger- and pose-aware tactile compressor, and injects the tactile latent into a video diffusion world model for joint visuo-tactile world modeling is presented.
Insertion tasks involve delicate, contact-rich manipulations commonly performed by robots in industrial and domestic environments. Vision-based insertion policies have shown progress, but they often become unreliable once contact is established, as the robot, object, or fixture can occlude the visual observations neede...
Yuan-Ji Li, Ming-Chao Qi, Jin-Hui Fan et al.· IEEE Robotics and Automation...· 0 citations
Two versions of a DexUMI-family exoskeleton are studied that share the same robot command definition, mapping procedure, and tactile module type but differ in hand-side geometry, showing that wearable collection hardware is part of the data-generation process for robot learning.
Rui-Tong Tian, Fang Xu, Noah B. Wilson et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.