The results show that spatial integration is implemented through late, feedback-driven processes operating on initially independent sensory estimates, resolving a long-standing discrepancy between early multisensory interactions and late spatial ventriloquism.
Abstract
During audiovisual perception, spatial information from vision and audition is combined, often producing biases such as the ventriloquist effect. While these interactions are well documented behaviorally, it remains unclear when cross-modal information begins to alter modality-specific spatial representations in the brain. Here we used cross-generalized inverted encoding modeling of electroencephalography (EEG) data to track the temporal evolution of spatial representations during a spatial ventriloquist task. Human participants (both sexes) localized audiovisual stimuli with horizontally offset auditory and visual components. Decoders trained on unisensory EEG responses were applied to audiovisual trials to estimate whether and when spatial representations of one modality were biased toward the other. Behaviorally, participants integrated cues but showed slight visual overweighting. Neural decoding revealed robust spatial representations for both modalities, with early unisensory encoding remaining unaffected by cross-modal input. Cross-modal biases emerged only later (from ∼200 ms onward), within a generalized representational window, and occurred sequentially: auditory representations were biased earlier than visual. These findings indicate that multisensory spatial integration arises via recurrent feedback during later processing stages, while early sensory-specific representations remain independent, providing a neural basis for the temporal dynamics underlying the ventriloquist effect.
Background: Visual imagery (VI) is a cognitive process that generates visual representations in the absence of external visual input; however, the spatiotemporal organization of its neural representations remains poorly understood. Methods: In this study, we combined electroencephalography (EEG) and functional magnetic...
Kai-Xin Huang, Lei Zhao, Fan Wang et al.· Brain Science· 0 citations
How does the auditory system implement spatial selection without a dedicated cortical map for space? One hypothesis holds that auditory spatial attention draws on a supra-modal network including the parietal-occipital cortex; an alternative implicates subcortical structures (e.g., superior colliculus) with no direct re...
Internal and external noise produce uncertainty in the neural representations of sensory stimuli. Uncertainty about basic stimulus features can be decoded from responses in sensory brain regions by estimating full probability distributions over stimulus features, rather than point estimates. Such decoded uncertainty co...
J. Nestor, Karen J. Tian, Angus F. Chapman et al.· bioRxiv· 0 citations
Neural representations of acoustic features that are key to speech perception have been studied extensively within the auditory cortex (AC), but their representations in higher-order cortical regions remain poorly understood. This study investigates the cortical encoding of three acoustic features in spoken vowels to d...
Yong-Tian Ou, K. Kay, A. Oxenham· Journal of Neuroscience· 0 citations
Face familiarity elicits reliable neural signatures during perception, yet how it is represented when faces are recalled from memory by using visual imagery and how these signals relate to perceptual representations, remains unclear. We combined perceptual and imagery conditions with electroencephalography and multivar...
Yeliz Dinç, Yang-Chun Shi, Gyula Kovács· bioRxiv· 0 citations
Attention can spread across sensory modalities so that when attention is directed to the visual component of a multisensory object, its task-irrelevant auditory component will also be bestowed with enhanced processing, which operates in either a stimulus-driven or representation-driven manner contingent on whether the...
Song Zhao, Fang-Fang Ma, Yu-Xin Zhou et al.· Cortex; a journal devoted to...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.