Aiden Dsouza

Topographic Training Concentrates Causal Circuits Without Improving Neuron Monosemanticity

Mechanistic interpretability of vision transformers seeks to decompose model computation into human-readable units, but learned representations entangle many concepts in each neuron. Feature superposition is widely treated as the central obstacle to this decomposition, yet most mitigations (sparse autoencoders, diction...