Seeing Through Touch: Visual Reliability Shapes Tactile Guidance in 360° Virtual Reality

Abstract

This study characterizes how people combine visual and tactile directional cues while acting in a fully immersive 360° virtual environment. Participants used a vibrotactile belt and VR headset to localize targets while we manipulated visual reliability and the spatial discrepancy between visual and tactile signals. Behaviorally, degraded visual input made visual responses slower, less precise, and more susceptible to tactile pull, whereas tactile-guided responses remained comparatively stable. We then asked whether these behavioral changes reflected a change in multisensory binding or a change in sensory uncertainty. A Bayesian Causal Inference (BCI) framework captured the structure of behavior under high visual reliability and continued to track individual differences under low visual reliability, even though its absolute goodness-of-fit decreased. Under extreme visual noise, Bayesian Information Criterion sometimes favored a simpler Maximum Likelihood Estimation (MLE) model, but MLE showed poor absolute fit and did not capture meaningful behavioral variability. This dissociation shows that statistical parsimony and explanatory validity can diverge when behavior becomes highly variable. BCI-derived parameters further indicated that degraded vision increased visual uncertainty, while the prior tendency to bind visual and tactile cues remained stable. Kinematic analyses added a complementary insight: early movement trajectories were strongly shaped by tactile signals, even when final localization was visually guided. Together, these findings suggest that visual–tactile integration in 360° environments depends on sensory reliability and task demands, with tactile cues providing fast body-centered guidance when visual information is limited.