Skip to content
Preprint

FlatClip: Reusing Image Foundation Models for fMRI Representation Learning via Cortical Flatmaps

Sep 2026 · 0 citations · 38 references
Computer Science Biology

TL;DR

FlatClip is introduced, which renders cortical activity as anatomy-linked flatmaps and extracts features with a frozen SigLIP2 encoder, which demonstrates a practical route for reusing image-pretrained features in fMRI and highlights cortical arrangement and input-region selection as important design choices.

Abstract

Can the spatial organization of cortical activity help transfer frozen image representations to fMRI prediction? We introduce FlatClip, which renders cortical activity as anatomy-linked flatmaps and extracts features with a frozen SigLIP2 encoder. Only a lightweight downstream probe is trained, without fMRI-specific encoder pretraining or fine-tuning. Across four resting-state benchmarks, FlatClip outperforms the evaluated ROI-level baselines on HCP and ADNI, but is weaker on PPMI and below the strongest voxel-level models overall. On NSD visual-fMRI decoding, restricting inputs to visual or NSD-provided task-active cortex improves recognition over whole-cortex inputs. Spatial perturbations reduce performance with both retrained and fixed readouts, and anatomy-linked arrangements outperform cortical-pixel permutations across three colormaps. Together, these results demonstrate a practical route for reusing image-pretrained features in fMRI and highlight cortical arrangement and input-region selection as important design choices. Code is available at https://github.com/OneMore1/FlatClip.

View source

Similar papers

Sep 2026

FlatClip: A Geometry-Aware Surface-Level Baseline for fMRI Representation Learning

FlatClip is introduced, a frozen-encoder surface-level baseline that renders cortical activity as geometry-aware flatmap sequences and reuses a frozen SigLIP2 image encoder with only a lightweight downstream probe, and support the utility of anatomy-linked spatial organization for reusing image-pretrained features.

Mo Wang, Wen-Hao Ye, Zi-Han Ning et al. · 0 citations
Open access Oct 2026

CortexDecoder: fMRI-to-image Reconstruction from Cortical Flatmaps

CortexDecoder, a brain decoding framework mapping fMRI flatmaps to images viewed by subjects achieves performance comparable to state-of-the-art brain decoding methods, demonstrating the effectiveness of flatmap based neural representations for visual reconstruction.

Teo Imoto-Tar, Maxwell Fung, Arnav Gurha · 0 citations
#machine learning Preprint Oct 2026

Latent Prediction Yields Cross-Subject Alignable Representations in Sensory-Evoked fMRI

Decoding sensory-evoked functional magnetic resonance imaging (fMRI) data is a central analytical tool in cognitive neuroscience. Here we ask whether self-supervised learning (SSL), which has been scarcely explored in this domain, can yield representations that support strong downstream performance. To do so, we adapt...

Serwan Jassim, Victoria Bosch, Daniel Anthes et al. · 0 citations
Preprint Oct 2026

CoFlu-Brain: An fMRI Connectivity Foundation Model with Cofluctuation-Based Pretraining

Functional magnetic resonance imaging (fMRI) commonly characterizes large-scale brain organization through functional connectivity (FC). However, an observed FC matrix is not a fixed brain graph but a statistical estimate determined by the finite BOLD frames used to compute it. Conventional FC-based approaches typicall...

Zi-Teng Sui, Jun-Feng Xia, Wen-Hao Ye et al. · 0 citations
Sep 2026

Natural Image Autoencoder-Based fMRI Representations for Trait and State Prediction

FReD is introduced, which derives fMRI representations from a frozen Deep Compression AutoEncoder pre-trained exclusively on natural images and pairs them with a task specific readout, making frozen natural-image features as a useful baseline for assessing its added value on current fMRI benchmarks.

Juhyeon Park, Yeonwook Kim, P. Y. Kim et al. · 0 citations

Flattening the Connectome Spectrum: A Spectral Filter for FC Induces a Pretraining Target for fMRI Encoders

It is shown that KRR is weighted by the eigenvalues of the FC which are miscalibrated for phenotype prediction, and an efficient spectral filter is applied to recalibrate the eigenvalues of each subject's FC matrix, enabling the model to exploit more inter-individual variance.

G. Marraffini, Victoria Shevchenko, Carlo Alberto Barbano et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.