NeuroGlyph, which learns a trial-independent visual target from multiple depths of a frozen visual backbone, achieves the strongest system-level performance in six of eight reported metrics, and support learning retrieval targets across the visual hierarchy rather than prescribing one visual depth.
Abstract
Brain-to-image retrieval seeks to identify the visual stimulus that elicited a non-invasive neural response. Candidate images are typically represented by pretrained vision models, whose internal representations vary in abstraction across depth. Existing methods usually train the neural encoder to recover a fixed final-layer visual target. Under this formulation, the visual hierarchy is reduced to a single prescribed endpoint, preventing representations at other depths from directly shaping the visual target. This limitation motivates learning how information across visual depths should contribute to the retrieval target. To this end, we introduce NeuroGlyph, which learns a trial-independent visual target from multiple depths of a frozen visual backbone. NeuroGlyph decomposes the target into factor-specific subspaces. Each subspace learns an image-conditioned allocation over visual depth. The resulting subspaces are fused into a single embedding for retrieval. Across THINGS-EEG and THINGS-MEG, NeuroGlyph outperforms final-layer supervision in all controlled comparisons. It also surpasses the post hoc best fixed-layer oracle in three of four comparisons. Parameter-matched ablations support both factorized target construction and image-conditioned depth allocation. Under comparable 200-way retrieval protocols, NeuroGlyph achieves the strongest system-level performance in six of eight reported metrics. These results support learning retrieval targets across the visual hierarchy rather than prescribing one visual depth.
This work proposes a training-free representation refinement that aligns frozen embeddings without updating either encoder, which preserves the spatial information of the Perception Encoder, converts its patch grid into a compact set of learned visual views, and aggregates them for each image with a block-structured, c...
The results provide a blueprint for how the brain encodes object concepts and suggest how concept representations engage perceptual representations through feedback during vision.
Spencer R. Loggia, Helen E. Feibes, S. Duffield et al.· Current Biology· 0 citations
Deep neural networks have achieved remarkable success in visual recognition; however, learning stable and discriminative visual representations under diverse imaging conditions remains a persistent challenge. Existing approaches often rely on increasing model depth or complex attention mechanisms, which improve perform...
Yu Wang, Ze-Yu Zhang, Y. Todo· International Conference on...· 0 citations
Deep neural networks predict neural responses across the visual hierarchy, yet published alignment scores do not reveal whether this correspondence reflects learned representations, architectural inductive biases, low-level image statistics, or categorical structure. We decompose DNN-brain alignment into four sources...
An ongoing challenge in sensory neuroscience is to characterize the feature dimensions encoded by cortical populations. Recent approaches probe feature selectivity in a data-driven way, by synthesizing a most-exciting-input (MEI) for a target neural population. While this approach has been successfully applied to human...
Jun-Ru Zhao, Han-Fei Guo, Andrew F. Luo et al.· arXiv.org· 0 citations
EEG-to-image retrieval commonly aligns neural responses with representations from pretrained vision encoders. However, these representations may underrepresent fine-grained visual details needed to distinguish semantically similar images. To address this limitation, we first investigate the EEG decodability and complem...
Ke Xiao, Bang-Hua Yang, Wen-Bin Guo et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.