This review surveys the full VBTS pipeline and treats sensing hardware, learning methods, simulation, and datasets as an integrated sensing-and-learning system to advance tactile intelligence for contact-rich robotic tasks.
Abstract
Tactile sensing is essential for robots in contact-rich tasks, yet many tactile sensors still provide sparse, low-dimensional signals that do not capture sufficient information for complex robotic perception and interaction. Vision-based tactile sensors (VBTSs) offer a powerful alternative by con-verting contact-induced deformation of a soft interface into im-ages. The image-based formulation gives VBTSs high-resolution, information-rich tactile observations that enable complex robotic tasks. This review surveys the full VBTS pipeline and treats sensing hardware, learning methods, simulation, and datasets as an integrated sensing-and-learning system. We 1) organize representative VBTSs into a hardware taxonomy structured by deformable elastomer design, sensor size and shape, and optical system design to guide future sensor development; 2) present a hierarchical view of learning-based tactile intelligence from low-level signal understanding to task-level policies and foundation models; and 3) examine simulation platforms and tactile datasets as a scaling layer, together with sim-to-real transfer and cross-sensor adaptation for training, benchmarking, and deployment. Finally, we identify open challenges and future directions for VBTSs in robotics. By providing a holistic view of how hardware, AI architectures, simulation, and datasets interact, this review aims to advance tactile intelligence for contact-rich robotic tasks.
Touch links perception to control by turning contact into actionable force. It provides pressure and friction cues that help robots regulate grip and execute precise manipulation, especially in unstructured environments. As robots move into open, real‐world settings, rigid tactile sensors struggle with surface conformi...
Vision-based tactile sensors (VBTSs) offer high-resolution, multimodal tactile sensing capabilities, supporting precise and dexterous robotic manipulation. However, the physical separation of sensing and perception modules necessitates frequent data transfer, introducing communication overhead and potentially constra...
Wen Fan, Jia-Jian Zheng, Ya-Nan Liu et al.· Communications Engineer· 1 citation
GlowTact, a pressure-responsive vision-based tactile sensing mechanism that directly visualizes contact pressure, is introduced, combining a durable nitrile membrane and simple optical design with sensitive and information-rich tactile perception.
Yu-Xiang Ma, Megha Tippur, Peng-Fei Ye et al.· arXiv.org· 0 citations
Tactile-JEPA is an efficient self-supervised pre-training method that uses the spatial arrangement of tactile sensors to learn topology-aware representations, and is trained to predict the embeddings of masked sensing elements from the unmasked remainder, using the sensor connectivity graph to guide spatial masking.
E. Kovtun, M. Konovalov, Andrey Sakhovskiy et al.· arXiv.org· 1 citation
We present $N_0$-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale multimodal data, tactile representation learning, and standardized evaluation. First, we engineer the infrastructure for scalable data collection, including a vision-based tactile se...
Tactile sensing is increasingly being incorporated into learning-based robotic manipulation, yet many existing approaches rely on spatially distributed sensors. Here we introduce {SpectRobot}, a framework that transforms single-point tactile signals into compact time-frequency spectrograms. These spectrograms encode hi...
Joseph Rigal, E. Virot, Caroline Pascal· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.