It is found that alignment with human fMRI, EEG, and macaque electrophysiology is already largely present at initialization, when networks classify at chance, and reaches a plateau within one to five epochs; thereafter it changes only modestly while classification accuracy continues to climb to 75%.
Abstract
Deep convolutional neural networks are leading models of biological vision, largely because of their strong brain alignment: their features predict neural responses better than earlier models. Yet they are believed to recognize objects differently, relying on texture where humans rely on shape and failing on perturbations humans handle effortlessly. What then does alignment reflect? Tracking three architectures densely during training, we find that alignment with human fMRI, EEG, and macaque electrophysiology is already largely present at initialization, when networks classify at chance, and reaches a plateau within one to five epochs; thereafter it changes only modestly while classification accuracy continues to climb to 75%. Network lesioning shows that a kernel’s contribution to alignment is essentially uncorrelated with its contribution to classification throughout training. Brain-network alignment therefore appears to reflect the structure of the visual environment both systems encode, rather than a shared solution for classification.
How neural activity across the ventral visual hierarchy supports face recognition is an open question. A long-standing debate asks whether face processing, particularly in fusiform cortex, relies on face-specific computations or representations shared with broader visual recognition. Here we combine source-resolved mag...
Hamza Abdelhedi, Shahab Bakhtiari, Karim Jerbi· bioRxiv· 0 citations
It is proposed that CNN training's classification bottleneck compresses brain-relevant information at depth, unlike transformers's self-attention and non-classification objectives, which reflect signal strength and persistence rather than distinct brain regions.
S. Baghel, Kshitij Dwivedi, Dinesh Singh et al.· arXiv.org· 0 citations
Deep neural networks predict neural responses across the visual hierarchy, yet published alignment scores do not reveal whether this correspondence reflects learned representations, architectural inductive biases, low-level image statistics, or categorical structure. We decompose DNN-brain alignment into four sources...
DRUMS outperforms Multi-modal Surface Matching (MSM), the current state-of-the-art method used in the Human Connectome Project (HCP) pipelines, over a wide range of input regularization settings in both registration accuracy and test-retest reproducibility of registration results.
Yu-Bo Wang, T. Coalson, Chun-Hui Yang et al.· bioRxiv· 0 citations
This work introduces axis-aligned feature accentuation, which converts each model’s fitted encoding axis into graded stimulus perturbations that are predicted to parametrically control neural firing within and beyond the natural-image range.
Jacob S. Prince, Binxu Wang, Thomas Fel et al.· bioRxiv· 0 citations
The results identify progressive integration of motion into object representations as a principle of robust dynamic vision and implicate predictive learning as a promising route toward realizing this computation in artificial systems.
Matteo Dunnhofer, Christian Micheloni, Kohitij Kar· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.