Jul 2026
A Compositional Theory of Causally Masked Transformers
This work develops an algebraic formalization that derives expressivity directly from the model's implemented dynamics, and obtains an expressivity hierarchy governed by the attention type under specific numerical semantics.
Franz Nowak, Ryan Cotterell, Reda Boumasmoud
· arXiv.org · 0 citations