Transformers for fMRI analysis: from spatiotemporal modeling to brain foundation models

Abstract

Functional magnetic resonance imaging (fMRI) serves as a pivotal tool for investigating the functional organization and dynamic connectivity of the human brain, indirectly reflecting neural activity through blood oxygen level-dependent (BOLD) signals. However, the high-dimensional spatiotemporal structure, prominent noise interference, and inter-individual variability of fMRI data pose significant challenges for traditional modeling methods in characterizing complex brain functional patterns. In recent years, the Transformer architecture has gained significant traction in fMRI data analysis, leveraging its self-attention mechanism to model global dependencies. It has demonstrated superior performance in tasks such as brain disease classification, functional connectivity modeling, and brain representation learning. Compared to convolutional and recurrent neural networks, Transformers can more effectively capture long-range spatiotemporal dependencies and offer stronger structural scalability and cross-task transfer potential. To systematically map the development of this field, this paper proposes a unified taxonomy for Transformer-fMRI methods based on modeling dimensions. It categorizes existing research into four primary paradigms: temporal modeling, spatial modeling, spatiotemporal joint modeling, and self-supervised learning, providing a comparative analysis of their structural characteristics and applicable scenarios. Furthermore, this paper summarizes the key challenges regarding computational complexity, data dependency, and interpretability, while exploring the integration of neuroscientific priors with large-scale pre-trained models. Finally, we discuss future directions for Transformers in the construction of brain foundation models and their clinical utility.