Skip to content
Preprint

VISTA: Visually Inferred Spatial ConTact Attention for Contact-Rich Manipulation

Aug 2026 · 0 citations · 42 references
Computer Science

Abstract

Contact-rich manipulation requires precise interaction feedback. While vision-centric imitation learning is prevalent, external visual observations provide indirect and ambiguous cues about contact states, particularly under occlusion or subtle object--gripper interactions; dedicated tactile or force sensors can provide rich contact information but introduce additional hardware complexity, calibration requirements, and deployment costs. To bridge this gap, we propose VISTA-Policy, an imitation learning paradigm that utilizes the Visual Deformation Field (VDF), a 3D displacement representation of a compliant gripper, as high-dimensional visuo-physical feedback. The framework integrates: 1) a Physics-Aware Encoding Engine for real-time VDF decoding; 2) an Energy Aggregation Denoising Mechanism to isolate true interaction signals; and 3) a Deformation-Augmented Policy Network with incremental gripper actions for precise closed-loop correction. Extensive evaluations on Cross-Scale Object Grasping, Cap Unscrewing, and Calligraphy Writing demonstrate that VISTA-Policy outperforms the strong pure-vision baseline 3D Diffusion Policy and the tactile baseline. VISTA-Policy further demonstrates substantial out-of-distribution generalization to unseen object scales and robustness against dynamic disturbances, offering a durable and cost-effective route toward general-purpose fine-grained manipulation in unstructured environments. Project videos and supplementary materials are available at: https://sites.google.com/view/vista-policy.

View source

Similar papers

Sep 2026

OccluDex: Hierarchical 3D Visuo-Tactile Representation Learning for Egocentric Dexterous Manipulation under Self-Occlusion

OccluDex is presented, a hierarchical 3D visuo-tactile representation learning framework that integrates global geometric structure with local contact information for robust manipulation under dynamic self-occlusion during hand-object interaction and could enable humanoid egocentric object manipulation for seen and uns...

Zi-Heng Xu, Yue-Yuan Chen, Xin-Yuan He et al. · 0 citations
Preprint Nov 2026

Visuo-tactile Diffusion Policy for Robotic Manipulation with Auxiliary Tactile Dynamics Prediction

Long-horizon robotic assembly requires reliable perception and action across grasping, alignment, and insertion. Vision captures global object pose and scene context, where as fingertip touch provides local contact cues that are especially useful as contact states change. We present a visuotactile diffusion policy that...

Kai-Hong Ouyang, Zi-Hao Su, Cheng-Xuan Zheng et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Equivariant Visual-Tactile Diffusion Policy for Contact-Rich Manipulation

Imitation learning for contact-rich manipulation requires high-quality expert data that is expensive to obtain. This makes learning a sample-efficient policy a key issue. To address this, we propose VISTA, a workspace-level equivariant visuotactile diffusion policy for data-efficient contact-rich imitation learning. VI...

L. H. Wong, Yi-Yao Ma, Xiu-Shen Wei et al. · 0 citations
Preprint Nov 2026

Semantics-to-Contact: A Stagewise Framework for Robust Contact-Rich Manipulation

Contact-rich manipulation requires precise control under complex contact dynamics while remaining robust to diverse visual conditions. However, real-world reinforcement learning often overfits to the visual appearance of training scenes. We propose Semantics-to-Contact (S2C), a stagewise framework for visually robust c...

Guan-Ren Qiao, Rui-Xiang Ouyang, Sheng-Qiang Xu et al. · 0 citations
Preprint Oct 2026

SimpleTouch: Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?

Tactile sensing provides essential contact information for robotic manipulation, yet incorporating it into pretrained vision-language-action (VLA) models remains challenging. A common concern is that simply introducing touch during task-specific fine-tuning may fail to bridge the cross-modal gap, yielding limited gains...

Chen Yang, Lin-Zhe Shi, Chang Jie Wu et al. · 1 citation · ⚡1
Sep 2026

PolyUMI: Accessible Visual-Tactile-Audio Data Collection for Object Inference and Manipulation

Humans typically rely on vision, touch, hearing, and proprioception to perceive contact and adapt their actions during manipulation. Providing robots with comparable responsiveness therefore requires hardware that can retain and use these complementary sensory signals. Most imitation-learning systems, however, observe...

Conor W. Hayes, Rickmer Krohn, Aravind Ramaswami et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.