A Unifying Framework for Concept-Based Representational Similarity

Abstract

Learned representations across models and modalities often exhibit striking structural similarities, suggesting shared underlying concept decompositions. However, concept alignment remains poorly defined: existing approaches optimize different objectives under the same terminology, obscuring what is actually aligned. We propose a unifying framework that decomposes alignment along two axes: what is aligned (representations vs. concepts) and at what level (instance-wise vs. distributional). This induces four corresponding properties---instance-wise and distributional variants of translation and concept consistency---and reveals precisely which of these guarantees existing methods provide. We further introduce InterVenchA, an intervention-based benchmark that separately measures extraction quality, translation quality, and concept consistency---which we validate against human perception of concept alignment. Using this framework, we show that commonly assumed equivalences between alignment objectives fail in practice: optimizing one property does not reliably recover the others, and, on real embeddings, purely unsupervised objectives fail to recover meaningful instance-level alignment. As a constructive consequence, a model, the Coupled SAE (CoSAE), matches or exceeds specialized alignment methods such as crosscoders and Universal SAEs on their own target metrics, by jointly enforcing the properties identified by our framework without any architectural innovation. Moreover, as little as 1% paired data is sufficient to recover instance-level alignment when anchoring distributional objectives. Overall, concept alignment is fundamentally multi-objective, and progress requires treating it as such. Our framework and benchmark provide the means to define, measure, and compare concept alignment methods on this basis.