Relational Attention for Data-Efficient Language Modeling
Relational BabyLM is a system submission to the BabyLM 2026 challenge that combines two cognitively motivated inductive biases in a single decoder-only Transformer, and introduces a novel symbol-retrieval mechanism that matches learned symbol libraries while adding no parameters.
Adrian Brasoveanu, Ece Takmaz, Jakub Dotlacil
· 0 citations