Skip to content
Preprint

EgoTac: In-the-wild Tactile Prediction from Egocentric Vision

Aug 2026 · 1 citation · ⚡ 1 influential · 48 references
Computer Science

TL;DR

Overall, EgoTac provides a scalable pathway to extract tactile priors from egocentric human videos, enabling broadly applicable tactile-aware robot learning.

Abstract

Touch is fundamental to dexterous manipulation, yet most egocentric human data increasingly used for robot learning lacks tactile information. Directly collecting large-scale tactile data is challenging due to sensor limitations, while human video data is abundant, contact-rich, and easily scalable. This motivates a natural question: can tactile signals be inferred purely from vision? To address this, we introduce EgoTac, a generalizable model that predicts rich tactile information directly from egocentric human videos. EgoTac is trained on a unified corpus of over 5.7M image-tactile pairs, covering both continuous force measurements and binary contacts. By learning from this diverse dataset, EgoTac captures nuanced touch dynamics across varied interactions. Experiments demonstrate strong performance: in-domain prediction achieves an average force error below 0.06N. On out-of-domain contact prediction benchmarks, EgoTac consistently outperforms the state-of-the-art contact estimator. It also captures the rise and fall patterns of real tactile data and enables zero-shot predictions on unconstrained real-world videos. Scaling analyses further reveal that both data diversity and volume improve performance steadily. Overall, EgoTac provides a scalable pathway to extract tactile priors from egocentric human videos, enabling broadly applicable tactile-aware robot learning.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

TouchSight: Bare-Handed Tactile Prediction from Egocentric Video via Generative Visual Augmentation

This work presents TouchSight, a monocular egocentric vision framework for dense full-hand contact force prediction that leverages 500 hours of pressure-glove recordings and extensive hand-object interaction (HOI) data and demonstrates that dense tactile signals can be recovered from egocentric vision alone, without ta...

Dan-Yan Zhou, Jin-Xuan Lu, Jia-Wei Lin et al. · 1 citation

DexTacWAM: A Visuo-Tactile World-Action Model for Dexterous Manipulation

A visuo-tactile WAM that encodes each fingertip independently, aggregates the resulting features through a finger- and pose-aware tactile compressor, and injects the tactile latent into a video diffusion world model for joint visuo-tactile world modeling is presented.

Hao-Ran Yuan, Ze-Kai Wang, Bo-Ning Shao et al. · 1 citation
Preprint Sep 2026

TACTIC: Understanding Tactile Encoders and Conditioning for Contact-rich Robot Manipulation Policies

Tactile information is essential for contact-rich manipulation tasks in robotics. Vision-based tactile sensors make it particularly easy to design end-to-end manipulation policies with tactile sensing, as they enable the use of existing encoders from computer vision. However, this has led to a huge variety of architect...

Seongjin Bien, Débora Oliveira Makowski, Carlo Kneissl et al. · 0 citations
Preprint Sep 2026

STAR: Sparse Tactile Representation Learning in Vision-Tactile-Language-Action Models for Dexterous Manipulation

Dexterous manipulation requires coordinated multi-finger control and effective tactile feedback, yet learning these capabilities remains challenging due to the lack of large-scale real-world data and the difficulty of extracting effective representations from sparse tactile signals. We build a robot platform and teleop...

Xiang-Cheng Liu, Tian-Hao Wu, Le Zheng et al. · 1 citation

Dexterous Tactile World Model

World models for manipulation are typically trained from video, yet the events that determine how manipulation unfolds, such as making and releasing contact, are difficult to observe visually and are often easier to sense through touch. We present the Dexterous Tactile World Model (DTWM), a video world model for future...

Zi-Yao Zeng, Xiatao Sun, Hao Wang et al. · 1 citation
#artificial intelligence Preprint Sep 2026

Unified Visual-Tactile-Action Modeling from Human Demonstrations for Dexterous Manipulation

A Unified Visual-Tactile-Action Model is proposed that maps both embodiments into aligned tactile and action representations and jointly predicts future action and tactile trajectories and achieves an average success rate of 70%, outperforming the strongest visual-tactile baseline, which achieves 29%, and an architectu...

Wen-Qiao Li, Qian-You Zhao, Jia-Wen Hao et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.