Conference
Open access
2026
Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse Autoencoders
This work proposes the Pairwise Dictionary Mean Correlation Coefficient (PW-MCC) as an assignment-based metric to quantify consistency and demonstrates that high levels are achievable with appropriate architectural choices for TopK SAEs on LLM activations with appropriate architectural choices.
Xiangchen Song, Aashiq Muhamed, Yujian Zheng et al.
· Annual Meeting of the Associ... · 0 citations