Skip to content
Review

Vision-Based Tactile Intelligence for Robotics: Sensing, Learning, and Embodied Manipulation

Aug 2026 · 2 citations · 237 references
Computer Science

TL;DR

This review surveys the full VBTS pipeline and treats sensing hardware, learning methods, simulation, and datasets as an integrated sensing-and-learning system to advance tactile intelligence for contact-rich robotic tasks.

Abstract

Tactile sensing is essential for robots in contact-rich tasks, yet many tactile sensors still provide sparse, low-dimensional signals that do not capture sufficient information for complex robotic perception and interaction. Vision-based tactile sensors (VBTSs) offer a powerful alternative by con-verting contact-induced deformation of a soft interface into im-ages. The image-based formulation gives VBTSs high-resolution, information-rich tactile observations that enable complex robotic tasks. This review surveys the full VBTS pipeline and treats sensing hardware, learning methods, simulation, and datasets as an integrated sensing-and-learning system. We 1) organize representative VBTSs into a hardware taxonomy structured by deformable elastomer design, sensor size and shape, and optical system design to guide future sensor development; 2) present a hierarchical view of learning-based tactile intelligence from low-level signal understanding to task-level policies and foundation models; and 3) examine simulation platforms and tactile datasets as a scaling layer, together with sim-to-real transfer and cross-sensor adaptation for training, benchmarking, and deployment. Finally, we identify open challenges and future directions for VBTSs in robotics. By providing a holistic view of how hardware, AI architectures, simulation, and datasets interact, this review aims to advance tactile intelligence for contact-rich robotic tasks.

View source

Similar papers

Review Open access Aug 2026

Flexible Sensors for Robotics Tactile Perception: A Review

Touch links perception to control by turning contact into actionable force. It provides pressure and friction cues that help robots regulate grip and execute precise manipulation, especially in unstructured environments. As robots move into open, real‐world settings, rigid tactile sensors struggle with surface conformi...

Yu Song, Ying Chen, Yi-Hao Chen et al. · 0 citations
Open access Sep 2026

A vision-based tactile sensor with in-sensor computing paradigm for seamless tactile sensing and perception

Vision-based tactile sensors (VBTSs) offer high-resolution, multimodal tactile sensing capabilities, supporting precise and dexterous robotic manipulation. However, the physical separation of sensing and perception modules necessitates frequent data transfer, introducing communication overhead and potentially constra...

Wen Fan, Jia-Jian Zheng, Ya-Nan Liu et al. · 1 citation
Sep 2026

GlowTact: Simple and Compact Vision-Based Tactile Sensing with High Sensitivity and Spatial Resolution

GlowTact, a pressure-responsive vision-based tactile sensing mechanism that directly visualizes contact pressure, is introduced, combining a durable nitrile membrane and simple optical design with sensitive and information-rich tactile perception.

Yu-Xiang Ma, Megha Tippur, Peng-Fei Ye et al. · 0 citations

Tactile-JEPA: Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors

Tactile-JEPA is an efficient self-supervised pre-training method that uses the spatial arrangement of tactile sensors to learn topology-aware representations, and is trained to predict the embeddings of masked sensing elements from the unmasked remainder, using the sensor connectivity graph to guide spatial masking.

E. Kovtun, M. Konovalov, Andrey Sakhovskiy et al. · 1 citation
Preprint Aug 2026

$N_0$-Foundation: Towards the Age of Tactile Intelligence

We present $N_0$-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale multimodal data, tactile representation learning, and standardized evaluation. First, we engineer the infrastructure for scalable data collection, including a vision-based tactile se...

NeoteAI Team, Fudan Teai Team · 4 citations

Learning tactile perception from high-bandwidth single-point sensing

Tactile sensing is increasingly being incorporated into learning-based robotic manipulation, yet many existing approaches rely on spatially distributed sensors. Here we introduce {SpectRobot}, a framework that transforms single-point tactile signals into compact time-frequency spectrograms. These spectrograms encode hi...

Joseph Rigal, E. Virot, Caroline Pascal · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.