Preprint
Aug 2026
TextNCA: Neural Cellular Automata for Language Modeling via Hierarchical Local Attention
Can a strictly local, iterated, weight-shared computation primitive support language modelling, and which of those three properties actually drives the model's behaviour? We define \textsc{TextNCA}, a 1D causal windowed-attention realisation of the Neural Cellular Automaton primitive, and study a hierarchical variant t...
Avni Mittal, Avinash Anand, Ashutosh Kumar et al.
· 0 citations