A new model family (CortexMAE) trained using the masked autoencoder framework is introduced, and in this setting the CortexMAE family outperforms prior models by a large margin, and the first open evaluation suite (Brainmarks) for fMRI foundation models is released.
Abstract
We study the problem of training self-supervised foundation models for functional MRI. Our main contributions are: (1) we introduce a new model family (CortexMAE) trained using the masked autoencoder framework on 2.1K hours of open fMRI data, and (2) we release the first open evaluation suite (Brainmarks) for fMRI foundation models. Our core innovation is simple: we adapt the Vision Transformer to fMRI by first converting each 3D fMRI volume to a 2D map using a cortical flat map projection. We directly compare flat maps to both parcellation and volume-based representations. While each has its advantages, flat maps generally perform best. We perform the first systematic scaling analysis for fMRI and observe strict power law scaling, albeit with limits. Finally, we use Brainmarks to do controlled benchmark comparisons. On subject-level trait prediction, we report a challenging null result: no single model achieves clear state-of-the-art performance. Moreover, all models struggle to outperform a simple functional connectivity baseline. On cognitive state decoding, we observe more robust performance, and in this setting our CortexMAE family outperforms prior models by a large margin. Code, models, and datasets are available at https://github.com/MedARC-AI/CortexMAE and https://github.com/MedARC-AI/Brainmarks.
It is shown that KRR is weighted by the eigenvalues of the FC which are miscalibrated for phenotype prediction, and an efficient spectral filter is applied to recalibrate the eigenvalues of each subject's FC matrix, enabling the model to exploit more inter-individual variance.
G. Marraffini, Victoria Shevchenko, Carlo Alberto Barbano et al.· arXiv.org· 0 citations
It is shown that MnemoDyn is compute efficient and generalizes very well across diverse populations and scanning protocols, and when benchmarked against current state-of-the-art transformer-based approaches, MnemoDyn consistently delivers superior reconstruction quality.
Sourav Pal, V. Lương, Hoseok Lee et al.· 1 citation
FlatClip is introduced, a frozen-encoder surface-level baseline that renders cortical activity as geometry-aware flatmap sequences and reuses a frozen SigLIP2 image encoder with only a lightweight downstream probe, and support the utility of anatomy-linked spatial organization for reusing image-pretrained features.
Mo Wang, Wen-Hao Ye, Zi-Han Ning et al.· arXiv.org· 0 citations
Functional magnetic resonance imaging (fMRI) commonly characterizes large-scale brain organization through functional connectivity (FC). However, an observed FC matrix is not a fixed brain graph but a statistical estimate determined by the finite BOLD frames used to compute it. Conventional FC-based approaches typicall...
Zi-Teng Sui, Jun-Feng Xia, Wen-Hao Ye et al.· 0 citations
A unified taxonomy for Transformer-fMRI methods based on modeling dimensions is proposed, which categorizes existing research into four primary paradigms: temporal modeling, spatial modeling, spatiotemporal joint modeling, and self-supervised learning, providing a comparative analysis of their structural characteristic...
FReD is introduced, which derives fMRI representations from a frozen Deep Compression AutoEncoder pre-trained exclusively on natural images and pairs them with a task specific readout, making frozen natural-image features as a useful baseline for assessing its added value on current fMRI benchmarks.
Juhyeon Park, Yeonwook Kim, P. Y. Kim et al.· arXiv.org· 0 citations
Known for his clear and elegant writing style, Bertsekas shaped fields from control and optimization to large-scale computation and artificial intelligence.