FlatClip is introduced, which renders cortical activity as anatomy-linked flatmaps and extracts features with a frozen SigLIP2 encoder, which demonstrates a practical route for reusing image-pretrained features in fMRI and highlights cortical arrangement and input-region selection as important design choices.
Abstract
Can the spatial organization of cortical activity help transfer frozen image representations to fMRI prediction? We introduce FlatClip, which renders cortical activity as anatomy-linked flatmaps and extracts features with a frozen SigLIP2 encoder. Only a lightweight downstream probe is trained, without fMRI-specific encoder pretraining or fine-tuning. Across four resting-state benchmarks, FlatClip outperforms the evaluated ROI-level baselines on HCP and ADNI, but is weaker on PPMI and below the strongest voxel-level models overall. On NSD visual-fMRI decoding, restricting inputs to visual or NSD-provided task-active cortex improves recognition over whole-cortex inputs. Spatial perturbations reduce performance with both retrained and fixed readouts, and anatomy-linked arrangements outperform cortical-pixel permutations across three colormaps. Together, these results demonstrate a practical route for reusing image-pretrained features in fMRI and highlight cortical arrangement and input-region selection as important design choices. Code is available at https://github.com/OneMore1/FlatClip.
FlatClip is introduced, a frozen-encoder surface-level baseline that renders cortical activity as geometry-aware flatmap sequences and reuses a frozen SigLIP2 image encoder with only a lightweight downstream probe, and support the utility of anatomy-linked spatial organization for reusing image-pretrained features.
Mo Wang, Wen-Hao Ye, Zi-Han Ning et al.· arXiv.org· 0 citations
CortexDecoder, a brain decoding framework mapping fMRI flatmaps to images viewed by subjects achieves performance comparable to state-of-the-art brain decoding methods, demonstrating the effectiveness of flatmap based neural representations for visual reconstruction.
Decoding sensory-evoked functional magnetic resonance imaging (fMRI) data is a central analytical tool in cognitive neuroscience. Here we ask whether self-supervised learning (SSL), which has been scarcely explored in this domain, can yield representations that support strong downstream performance. To do so, we adapt...
Serwan Jassim, Victoria Bosch, Daniel Anthes et al.· 0 citations
Functional magnetic resonance imaging (fMRI) commonly characterizes large-scale brain organization through functional connectivity (FC). However, an observed FC matrix is not a fixed brain graph but a statistical estimate determined by the finite BOLD frames used to compute it. Conventional FC-based approaches typicall...
Zi-Teng Sui, Jun-Feng Xia, Wen-Hao Ye et al.· 0 citations
FReD is introduced, which derives fMRI representations from a frozen Deep Compression AutoEncoder pre-trained exclusively on natural images and pairs them with a task specific readout, making frozen natural-image features as a useful baseline for assessing its added value on current fMRI benchmarks.
Juhyeon Park, Yeonwook Kim, P. Y. Kim et al.· arXiv.org· 0 citations
It is shown that KRR is weighted by the eigenvalues of the FC which are miscalibrated for phenotype prediction, and an efficient spectral filter is applied to recalibrate the eigenvalues of each subject's FC matrix, enabling the model to exploit more inter-individual variance.
G. Marraffini, Victoria Shevchenko, Carlo Alberto Barbano et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.