Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders
The results suggest that architectural routing mechanisms may have negligible impact on core semantic understanding, with representational divergence confined to extreme structural margins.
Rithin Nagaraj, Rupa Laalasa Oruganti, Prerna Subhashchandra Kunder et al.
· 0 citations