Skip to content

What masking geometry works best for EEG foundation models?

Sep 2026 · 0 citations · 105 references
Computer Science

TL;DR

This paper formalizes the design choices for spatio-temporal masking strategies and train various models with a single pipeline under varying masking configurations across two SSL frameworks (MAE and JEPA), and systematically evaluates the resulting 58 pre-trained models under a linear probe.

Abstract

EEG foundation models hold promise for scalable brain-signal decoding across clinical and cognitive neuroscience applications, yet their pre-training pipelines remain poorly understood. Among design choices, the masking strategy is particularly critical: it determines what the network must predict and from which context. Yet it has never been ablated in isolation, as each new model bundles a new masking strategy with a new backbone and objective. In this paper, we formalize the design choices for spatio-temporal masking strategies and train various models with a single pipeline under varying masking configurations across two SSL frameworks (MAE and JEPA). We then systematically evaluate the resulting 58 pre-trained models on the 12 datasets of OpenEEGBench under a linear probe. Both frameworks agree on an optimal masking configuration and on shared failure modes. Outside these, performance is robust: 11 MAE and 9 JEPA configurations are statistically indistinguishable from the best. We further identify a novel JEPA-specific failure mode, tagged bias-inflation collapse, invisible to standard detectors. With a well-chosen mask, our pipeline reaches REVE-level downstream performance at a fraction of REVE's pre-training compute.

View source

Similar papers

#artificial intelligence Review Sep 2026

A Roadmap for MEG Foundation Models

This perspective lays down the basic concepts needed to understand MEG foundation models and provides a didactic overview of the field's key design choices, including tokenization, sensor- versus source-space representations, sensor-geometry encoding, backbone architectures, self-supervised objectives, and pretraining...

Philipp Thölke, Hamza Abdelhedi, Yorguin-José Mantilla-Ramos et al. · 1 citation
#machine learning Preprint Oct 2026

SPD-MetaFormer is what you need for small-data brain decoding

This work examines two representative architectures, MAtt and GBWAtt, and finds that their learned attention weights remain close to uniform after training, and introduces SPD-MetaFormer, an attention-free architecture built on uniformly weighted Fr\'echet aggregation under log-Euclidean geometry.

Zhi-Da Wang, Wei Lyu, Yu Guo et al. · 0 citations
Book Open access Oct 2026

Investigating Foundation Models, Disentanglement and Latent Alignment for Subject-Independent EEG Learning

This work proposes a modular framework for the unified evaluation of these paradigms in subject-independent learning, and conducts extensive experiments on Motor Imagery and Event-Related Potential datasets, representative of interactive BCI scenarios spanning both oscillatory (MI) and event-related (ERP) paradigms.

Lia Schmid, Jacopo Burger, Alessandro D'Amelio et al. · 0 citations

Matched-Input Estimates Differ in Sign Across Architectures: Auditing EEG Foundation Models on Motor Imagery

Sign differences suggest that a single comparator may not provide an architecture-invariant decomposition of a pretrained-versus-supervised performance gap, and validation-fitted temperature scaling returns foundation-model calibration error to the supervised range despite substantially lower four-class accuracy.

Ke-Tian Zhou, Sparsh Roy · 1 citation
Preprint Sep 2026

Scaling Laws for EEG Decoding: How Much Data Is Enough?

Deep learning has become a cornerstone of EEG-based brain decoding, with a growing number of architectures proposed every day. However, how the performance of these different models scales with data volume is not clear. Although this relationship has been characterized in other fields under the name of scaling laws, it...

José Manuel Nunes de Oliveira, Bruna J. Lopes, Léo Burgund et al. · 0 citations

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.