S. Vallisa

The Changing Geometry of Grammar: Dimensionality and Neighborhood Reorganization across Transformer Layers

This work investigates the layer-wise evolution of ID, and shows that closed-class items expand earlier and collapse sooner than open-class ones, and recovers a token's grammatical role, and uses them to interpret how the semantic content of each PoS evolves across layers in a downstream classification task.