Skip to content

Shaping Slow Manifolds in Equivariant Recurrent Neural Networks.

Sep 2026 · Neural Computation · pp. 1-69 · 0 citations
Medicine

TL;DR

This work introduces a new class of equivariant recurrent neural networks, where the connectivity is based on group convolution, and reduces such networks to low-rank models using the group Fourier transform.

Abstract

Recordings of increasingly large neural populations have revealed that the firing of individual neurons is highly coordinated. When viewed in the space of all possible patterns, the collective activity forms nonlinear structures called neural manifolds. Because such structures are observed even at rest or during sleep, an important hypothesis is that activity manifolds may correspond to continuous attractors shaped by recurrent connectivity between neurons. Classical models of recurrent networks have shown that continuous attractors can be generated by specific symmetries in the connectivity. Although a variety of attractor network models have been studied, general principles linking network connectivity and the geometry of attractors remain to be formulated. Here, we address this question by using group representation theory to formalize the relationship between the symmetries in recurrent connectivity and the resulting fixed-point manifolds. We start by revisiting the classical ring model, a continuous attractor network generating a circular manifold. Interpreting its connectivity as a circular convolution, we draw a parallel with feedforward convolutional neural networks. Building on principles of geometric deep learning, we then generalize this architecture to a broad range of symmetries using group representation theory. Specifically, we introduce a new class of equivariant recurrent neural networks, where the connectivity is based on group convolution. Using the group Fourier transform, we reduce such networks to low-rank models. This formulation gives us a low-dimensional nonlinear description that can be fully analyzed to determine the symmetry, dimensionality, and stability of fixed-point manifolds. Our results underline the importance of stability considerations: for a connectivity with a given symmetry, depending on parameters, several manifolds with different topology can coexist, some stable and others consisting of saddle points. Our framework unifies a variety of existing models and offers a principled approach to build new types of continuous attractor models of neural systems.

View source

Similar papers

Oct 2026

Symmetries and continuous attractors in disordered neural circuits.

This work builds, directly from head-direction cell data, recurrent networks that behave as quasicontinuous attractors, and recovers the ingredients of classical ring attractors, namely, Mexican hat interactions and a spontaneously broken symmetry producing bump states.

David G. Clark, L. Abbott, H. Sompolinsky · 1 citation
Oct 2026

Brain-inspired recurrent spiking neural networks for multiscale dynamic representation with manifolds.

Understanding how network structure shapes neural population dynamics and representation of external inputs is a key issue in neuroscience. This study proposes a modeling framework for recurrent spiking neural networks (RSNN) and elucidates dynamical computation mechanism of RSNN by linking input dimensionality, networ...

Xiao-Jun Wang, Zai-Dong Lin, Bin Deng et al. · 0 citations
Open access Oct 2026

Attractor dynamics underlie sequence working memory in data-constrained recurrent neural networks

Sequences of stimuli can be maintained in working memory by encoding each item, together with its position, as a distinct pattern of neural population activity. Competing hypotheses about the underlying dynamics have been proposed: patterns of activity might be supported either by stable attractor states that persist o...

Matthijs Pals, J. Macke · 0 citations
Preprint Aug 2026

Adaptive self-organized criticality in deep neural networks

This work shows that the global dynamical state of a deep neural network can be autonomously regulated by purely local homeostatic plasticity, and demonstrates how adaptive self-organization can be implemented in deep neural networks and how local plasticity can control their collective dynamical operating point.

Simon Vock, C. Meisel · 0 citations
Open access Sep 2026

A common structure in recurrent networks supports neural sequence generation locally and in downstream neurons

Neural sequences, characterized by neurons or groups of neurons that fire one after the other, have been observed in multiple brain regions, across species, and are known to underlie a diversity of brain functions. To flexibly support behaviour and cognition, neural sequences exhibit much variability in properties like...

Lea Marie Braun, Mathias Karsrud Nordal, Silje-Neneva Hanssen Rambø et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.