Geometry-Aware Visual-Tactile Image Representation Learning with Uniform Proxies on the Hypersphere

Abstract

Understanding the physical properties of object surfaces is crucial for robotic systems to perform delicate operations and perceive the physical environment. Recent advances in optical sensor-based tactile imaging enable the capture of high-resolution contact information, demonstrating significant potential for multimodal robotic perception. Despite this progress, visual-tactile image representation learning remains under-studied. Our analysis explores two key challenges: visual and tactile images exhibit significant spatial inconsistencies, and tactile data shows more severe inter-class similarity and intra-class variation than common modalities. These factors limit the effectiveness of existing approaches based on direct instance-level alignment. To address these challenges, we propose a geometry-aware visual-tactile representation learning framework on a hyperspherical space. Specifically, we introduce a set of modality-agnostic class proxies that serve as shared semantic anchors to guide cross-modal alignment. Instead of enforcing direct pairwise correspondence, both modalities are organized around these proxies, enabling more stable and consistent semantic grouping. Furthermore, we impose a uniformity constraint on the proxies to explicitly regularize the global structure of the hyperspherical space, improving class separability. Extensive experiments on multiple visual-tactile benchmarks demonstrate that the proposed method consistently outperforms existing approaches and leads to well-structured and discriminative hyperspherical representations.