Cortical encoding of probabilistic temporal predictions during speech perception

Abstract

The temporal structure of speech has traditionally been characterized by the rhythmicity of its canonical linguistic units (phonemes, syllables, words), each summarized by a mean occurrence rate. While valid, this view overlooks whether speech carries a finer, context-dependent and probabilistic temporal structure that could support temporal predictive coding during listening. Using large French and English speech corpora, we trained models of increasing complexity to predict the onsets of linguistic units. Recurrent neural networks (RNNs) outperform mean-rate and hazard-rate models, showing that the variability around these rates is not noise but a temporal structure shaped by local context, statistically predictable across phonemes, syllables and words. Recording from 7,698 intracerebral electrodes in 53 neurosurgical patients listening to natural speech, we next show that the models’ output), the continuous probability of an upcoming onset (when), explains neural activity beyond acoustic and linguistic content (what) features, with markedly stronger effects for RNNs than for mean- or hazard-rate models. This dynamic neural prediction of when an onset will occur is dissociable from the encoding of linguistic content, relying on largely distinct channel populations. Temporal predictions engage a distributed cortical network extending from bilateral temporal cortex into left frontal and sensorimotor regions. Together, these results establish temporal prediction in speech as a dynamic, context-dependent and probabilistic process in its own right. Significant Statement When we listen to speech, the brain must anticipate not only what comes next but when it will occur. The what has been modeled within a dynamic long-term context; the when has mostly been characterized by a static temporal context through its periodicity. Analysing large speech corpora, we show that the onsets of linguistic units are predictable beyond their mean rate, carrying contextual, dynamic and probabilistic information. Using intracerebral recordings in patients listening to spoken stories, we demonstrate that the brain continuously encodes this temporal information, in neural populations largely separate from those that respond to linguistic content. These results place temporal structure at the heart of speech prediction, extending predictive coding from what is said to when it unfolds.