2026· IEEE Transactions on Automation Science and Engineering· Vol 23, pp. 17805-17817· 0 citations· 40 references
TL;DR
The ST-Tacformer model is introduced, which extracts spatio-temporal features from different feature scales and resolves the issue of information loss caused by traditional deep learning methods when extracting single features.
Abstract
Dexterous hands equipped with tactile sensors not only enable flexible object manipulation but also significantly enhance object identification capabilities, particularly in vision-failure scenarios. However, the arrangement of tactile sensors within the dexterous hand leads to challenges such as difficult signal interpretation and significant signal variability, which ultimately hinder object identification accuracy. Therefore, in this work, we propose a bio-inspired tactile recognition system. This system consists of two main components: a distributed Event-driven Tactile Sensor (ET-Skin) and a Spatial-temporal coupling Tactile-Transformer network(ST-Tacformer). The ET-Skin is designed by neuronal triggering mechanisms, featuring flexibility, low-cost and rapid response. It works as pressure receptor with the ability to measure contact pressure values, which is integrated onto a multi-fingered robot hand to perceive object information by grasping operation. In order to organize tactile data into a coherent structure for feature extraction and object recognition, the ST-Tacformer is proposed. It utilizes topology graph to consolidate tactile inputs and learns a robust tactile feature representation with attention mechanisms. The ET-Skin exhibits high positional and force sensing accuracy. The performance and applicability of ST-Tacformer are evaluated on two public datasets and two self-collected datasets. Comparative studies show that our model significantly improves recognition accuracy, achieving improvements of 4.5% and 12.6% on the public datasets and accuracies of 95.67% and 93.06% on our self-collected datasets. Note to Practitioners—Robotic tactile perception is an important research area that can enhance the flexibility, accuracy and stability of robots in tactile manipulation tasks. However, the accuracy of tactile recognition relies on the performance of tactile sensors and the quality of the features extracted, which are limited by conventional deep learning methods. To address this, we propose a bio-inspired tactile perception system. The low-cost, scalable and event-driven characteristics of the proposed sensors can support various grasping tasks across different dexterous hands. We introduce the ST-Tacformer model, which extracts spatio-temporal features from different feature scales. This network resolves the issue of information loss caused by traditional deep learning methods when extracting single features. Experiments on two public datasets and two datasets collected by our proposed sensors show that our method outperforms competing algorithms in object recognition tasks in robotic multi-tactile sensor scenarios. In future research, we will focus on the characteristics of bio-inspired sensors and the stable grasping perception problem of dexterous hands.
In-hand manipulation of ultra-soft objects is challenging due to their high deformability. Conventional vision-based tactile sensors (VBTSs) with reflective coatings often fail to detect contact signals from ultra-soft items like sponges, since the sensing material is stiffer than the object itself, resulting in insuff...
Zhi-Tong Liu, Xin Jiang· IEEE Robotics and Automation...· 0 citations
A growing body of work suggests that tactile sensing gives robot policies contact information that complements vision in dexterous manipulation. However, visuo-tactile robot data remains scarce: dexterous demonstrations require teleoperating robots, which limits dataset scale. Human demonstrations are far cheaper to co...
Jing-Yun Yang, Bai-Yu Shi, Timothy Yu et al.· 0 citations
To perform these movements better, dexterous hands need to be able to feel contact with objects and the surrounding environment. This article mainly introduces tactile technology for dexterous hands, and it discusses these technologies from the view of embodied intelligence. The article mainly introduces five types of...
Xiao-Yi Gong· Applied and Computational En...· 0 citations
Human hands have rich tactile senses with multimodal and multidimensional perceptions for hand dexterous manipulation. However, current robotic tactile sensory modality and dimensionality are deficient and hardly meet the harsh demands for diverse manipulation tasks. Here, we report a multimodal and multidimensional ta...
Humanoid hands require tactile feedback across the whole finger, not just the fingertip, to grasp and manipulate objects properly. Vision and proprioception alone cannot reliably provide this information, particularly when the hand's own fingers occlude the camera's view of the grasp. We present a low-cost tactile arra...
Neel Adwani, Muhaiminul Islam Akash, Rituja Bhattacharya et al.· arXiv.org· 0 citations
Pre-touch sensing, or sensing information about an object before contact, is an attractive modality for robot hands as it can allow a robot to anticipate and respond to a contact before it happens. In this work, we introduce ToFTouch, a multimodal tactile sensor combining three multizone (4$\times$4 resolution) time-of...
Eric T. Chang, Bianca Aumann, Zhan-Peng He et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.