Skip to content

Scaling Vision Transformers for Functional MRI with Flat Maps

Oct 2025 · arXiv.org · Vol abs/2510.13768 · 5 citations · ⚡ 1 influential · 152 references
Computer Science Biology

TL;DR

A new model family (CortexMAE) trained using the masked autoencoder framework is introduced, and in this setting the CortexMAE family outperforms prior models by a large margin, and the first open evaluation suite (Brainmarks) for fMRI foundation models is released.

Abstract

We study the problem of training self-supervised foundation models for functional MRI. Our main contributions are: (1) we introduce a new model family (CortexMAE) trained using the masked autoencoder framework on 2.1K hours of open fMRI data, and (2) we release the first open evaluation suite (Brainmarks) for fMRI foundation models. Our core innovation is simple: we adapt the Vision Transformer to fMRI by first converting each 3D fMRI volume to a 2D map using a cortical flat map projection. We directly compare flat maps to both parcellation and volume-based representations. While each has its advantages, flat maps generally perform best. We perform the first systematic scaling analysis for fMRI and observe strict power law scaling, albeit with limits. Finally, we use Brainmarks to do controlled benchmark comparisons. On subject-level trait prediction, we report a challenging null result: no single model achieves clear state-of-the-art performance. Moreover, all models struggle to outperform a simple functional connectivity baseline. On cognitive state decoding, we observe more robust performance, and in this setting our CortexMAE family outperforms prior models by a large margin. Code, models, and datasets are available at https://github.com/MedARC-AI/CortexMAE and https://github.com/MedARC-AI/Brainmarks.

View source

Similar papers

Flattening the Connectome Spectrum: A Spectral Filter for FC Induces a Pretraining Target for fMRI Encoders

It is shown that KRR is weighted by the eigenvalues of the FC which are miscalibrated for phenotype prediction, and an efficient spectral filter is applied to recalibrate the eigenvalues of each subject's FC matrix, enabling the model to exploit more inter-individual variance.

G. Marraffini, Victoria Shevchenko, Carlo Alberto Barbano et al. · 0 citations
Preprint Aug 2026

MnemoDyn: Learning Resting State Dynamics from 40K FMRI sequences

It is shown that MnemoDyn is compute efficient and generalizes very well across diverse populations and scanning protocols, and when benchmarked against current state-of-the-art transformer-based approaches, MnemoDyn consistently delivers superior reconstruction quality.

Sourav Pal, V. Lương, Hoseok Lee et al. · 1 citation
Sep 2026

FlatClip: A Geometry-Aware Surface-Level Baseline for fMRI Representation Learning

FlatClip is introduced, a frozen-encoder surface-level baseline that renders cortical activity as geometry-aware flatmap sequences and reuses a frozen SigLIP2 image encoder with only a lightweight downstream probe, and support the utility of anatomy-linked spatial organization for reusing image-pretrained features.

Mo Wang, Wen-Hao Ye, Zi-Han Ning et al. · 0 citations
Preprint Oct 2026

CoFlu-Brain: An fMRI Connectivity Foundation Model with Cofluctuation-Based Pretraining

Functional magnetic resonance imaging (fMRI) commonly characterizes large-scale brain organization through functional connectivity (FC). However, an observed FC matrix is not a fixed brain graph but a statistical estimate determined by the finite BOLD frames used to compute it. Conventional FC-based approaches typicall...

Zi-Teng Sui, Jun-Feng Xia, Wen-Hao Ye et al. · 0 citations
Review Open access Oct 2026

Transformers for fMRI analysis: from spatiotemporal modeling to brain foundation models

A unified taxonomy for Transformer-fMRI methods based on modeling dimensions is proposed, which categorizes existing research into four primary paradigms: temporal modeling, spatial modeling, spatiotemporal joint modeling, and self-supervised learning, providing a comparative analysis of their structural characteristic...

Xiao-Min Yang, Yue Li, Xue Zhang et al. · 0 citations
Sep 2026

Natural Image Autoencoder-Based fMRI Representations for Trait and State Prediction

FReD is introduced, which derives fMRI representations from a frozen Deep Compression AutoEncoder pre-trained exclusively on natural images and pairs them with a task specific readout, making frozen natural-image features as a useful baseline for assessing its added value on current fMRI benchmarks.

Juhyeon Park, Yeonwook Kim, P. Y. Kim et al. · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.