Skip to content

Feature Superposition in Neural Networks: From Theory to Practice

Sep 2026 · 0 citations
Computer Science

TL;DR

This survey reviews the geometry, learning, and computation of superposed representations, explaining how feature statistics and decoder choice affect the conclusions and compares practical methods for recovering and analyzing features.

Abstract

Superposition refers to neural networks representing more features than they have dimensions. It offers a possible explanation for polysemantic neurons and motivates methods for recovering interpretable features from neural activations. Theoretical models typically start with a given set of input features and assumptions about how their values vary across inputs, then study how a network encodes those values in a lower-dimensional hidden representation. Empirical work, by contrast, seeks to identify the features encoded in trained networks and determine their role in computation. In this survey, we review the geometry, learning, and computation of superposed representations, explaining how feature statistics and decoder choice affect the conclusions. To connect these theoretical accounts with evidence from trained networks, we compare practical methods for recovering and analyzing features and examine what their evaluations establish. Since accurate activation reconstruction alone does not establish feature identity or causal use, we discuss the methods'documented failures and applications in light of the evidence available for these different claims. Finally, we assess previously stated open problems and identify remaining theoretical and empirical questions about superposition in trained networks. We hope our work can pave the way for a deeper understanding of superposition and more reliable methods for interpreting neural networks.

View source

Similar papers

Review Open access Aug 2026

Making models disagree to learn how brains compute

Computational hypotheses about brain information processing can be expressed in neural network models. Neuroscientists have begun to compare such models in terms of their alignment with neural and behavioural data. The high parametric capacity of these models is essential to their ability to capture cognitive processes...

Tal Golan, H. Schütt, N. Kriegeskorte · 0 citations
#machine learning Preprint Oct 2026

Generalization in Neural Networks Through the Lens of Magnitude Potential

This paper introduces the idea of {\em magnitude potential} -- a quantity based on the theory of metric magnitude -- that reflects how well an arbitrary point is represented by a given set, and finds that this basic quantity can be applied to examine various features in neural generalization.

Sahel Torkamani, Henry Gouk, Rik Sarkar · 0 citations
Sep 2026

Model for computing with population-encoded variables explains neural correlations

A computational model is proposed that carries out operations on population-encoded variables by carefully configuring the connections between neurons through dendritic branches, and predicts neural correlations as a byproduct of the computation, with correlation coefficients aligning with experimentally measured value...

Heiko Hoffmann · 0 citations
#machine learning Preprint Sep 2026

Parameter symmetries determine representational geometry in overparameterized nonlinear networks

Representations are routinely used across machine learning, psychology, and neuroscience to draw inferences about the computations of biological and artificial systems. Such inferences presume a meaningful link between representational geometry and the computation being performed. For artificial neural networks, howeve...

Marvin Theiss, Lukas Braun, Andrew M. Saxe et al. · 0 citations
Review Sep 2026

Direct codes in a distributed age.

It is suggested that the central challenge is not to choose between direct and distributed codes, but to determine when geometric descriptions illuminate mechanism and when they merely rename it.

M. Shadlen · 1 citation

Related blog posts

Microsoft Research Blog Nov 16, 2026

TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.