Jul 2026· Simulation (San Diego, Calif.)· 0 citations· 35 references
TL;DR
This paper integrates reinforcement learning with reward-modulated, bio-inspired SNNs that employ reward-modulated spike-timing-dependent plasticity (R-STDP) to address real-time control tasks and reconstructs the task-oriented state space and reward mechanism to realize a cross-task control framework.
Abstract
Spiking neural networks (SNNs), with temporal dynamics and biologically inspired spiking behavior, offer improved interpretability and performance in real-time control. This paper integrates reinforcement learning (RL) with reward-modulated, bio-inspired SNNs that employ reward-modulated spike-timing-dependent plasticity (R-STDP) to address real-time control tasks. The proposed model enables bio-inspired learning by combining conventional spike-timing-dependent plasticity (STDP) with time-dependent, reward-modulated weight updates. Furthermore, it reconstructs the task-oriented state space and reward mechanism to realize a cross-task control framework. Experimental results from control tasks, along with comparisons to traditional RL methods, demonstrate the effectiveness of SNNs in both simulation and real-world applications. This study aims to bridge the gap between SNNs and traditional feedback mechanisms in control systems, highlighting how bio-inspired approaches can enhance adaptive control.
This work proposes a noisy group neuron (NGN) model, which incorporates population-level synchronous resetting and neural stochasticity as fundamental computational mechanisms, and develops the NGN method as a framework that combines the NGN model with backpropagation learning based on mean-field dynamics.
Yajie Zhai, Yanmei Kang, Meng Li et al.· 0 citations
Background / Introduction: Biologically inspired spiking neural networks can model adaptive behavior, but learning multiple goals is difficult because synaptic updates for different targets can interfere. We tested whether multi-timescale plasticity and context-specific credit assignment could improve continual multi-goal learning in a spiking navigation system inspired by entorhinal-hippocampal circuitry. Methods: We developed a closed-loop spiking model containing grid-like, place-like, target-related, association, and motor-output populations. An agent navigated in a two-dimensional environment with randomized starting locations and learned through reward-modulated spike-timing dependent plasticity (STDP/RL) and a novel evidence-gated plasticity (EGP) framework. EGP accumulates candidate synaptic modifications, evaluates them using reward evidence, and consolidates only changes that improve performance. A target-context variant maintained separate proposal stores and reward evaluation for each target. Results: STDP/RL learned and retained a single-target navigation policy, but multi-target training produced substantial interference, including attraction to incorrect targets after learning. Across 10 connectivity seeds, target-context EGP achieved higher late-stage reward than global EGP, improved weakest-target performance, and increased the fraction of targets achieving positive reward. In a longer continual-learning simulation, reward increased for all targets, TEST-phase performance increasingly exceeded TRAIN-phase performance, and proposal magnitudes grew over learning. Dwell-time confusion analyses showed that target-context EGP reduced wrong-target attraction and improved target selectivity relative to multi-target STDP/RL. Conclusions: These results demonstrate that spiking navigation circuits can learn goal-directed behavior using local plasticity, but robust multi-goal learning benefits from context-specific evidence-based consolidation. Target-context EGP provides a biologically motivated mechanism for reducing interference during continual reinforcement learning in spiking neural networks.
Samuel A Neymotin, Hananel Hazan, Gozde Unal et al.· Research Square· 0 citations
It is aimed at proving that SNNs have potential in such areas as computer vision, robotics, and speech recognition, and their role in overcoming the barrier between artificial and biological neural systems is proved.
Mesala Sravani, K. Kumari, S. M. Reddy· International Journal of Unc...· 0 citations
This work presents a Hebbian local learning rule that models synaptic modification as a function of calcium traces tracking neuronal activity and demonstrates how spike timing and rate can be complementary in their role of shaping the connectivity of spiking neural networks.
Willian Soares Girāo, Nicoletta Risi, Caroline Geisler et al.· Neuromorphic Computing and E...· 0 citations
Reinforcement learning (RL) algorithms have made strides over the past decade applying them to a wide range of problems and control tasks. However, the deployment of RL on neuromorphic hardware for continuous control tasks remains under-validated. Namely it is unclear whether replacing a conventional actor network with a spiking neural network (SNN) affects the performance of an agent before any hardware-specific benefits manifest. We provide a systematic validation of a minimal, neuromorphically viable spiking actor variant of Soft Actor-Critic (SAC) on conventional hardware, establishing a baseline for future neuromorphic RL research. In this paper, we propose the Spiking Actor Network Soft Actor Critic (SANSAC) to address the use of RL frameworks in continuous environments, designed as a framework that can be implemented on neuromorphic hardware. We compare a traditional Soft Actor Critic (SAC) network to SANSAC in a traditional computer. We demonstrate the near equivalent performance of SANSAC and SAC, while addressing the impact of hidden dimensions. Our results demonstrate the viability of SNN based algorithms in complex continuous environments, as well as competitive performance to traditional neural networks in traditional computers, providing a basis to continue exploring the use of SNNs in continuous RL frameworks.
Spiking neural networks (SNNs) have garnered significant attention in reinforcement learning tasks for their low power consumption. However, traditional spiking reinforcement learning (SRL) methods, which rely on local-connected encoding and fixed-threshold learning, struggle to capture the inter-dimensional correlations of input information within short timesteps, limiting the network’s expressive capacity at low timesteps. While increasing timesteps can significantly enhance performance, excessive timesteps result in substantial delays. To address this contradiction and enhance the expressive and decision-making capabilities of SNNs within short timesteps, we propose Mask-Adaptive Global Connection (MAGC), a novel encoding method that efficiently captures long-range dependencies via sparse, adaptively masked connections—enabling global feature interaction in a single timestep. Additionally, dynamic-threshold spiking neurons are introduced to effectively capture and distinguish subtle changes in input signals at each timestep, thereby enhancing the spatial-temporal state representation during spike information transmission. Extensive experimental results demonstrate that the proposed method achieves performance comparable to state-of-the-art algorithms using only a single timestep, while significantly reducing inference latency and energy consumption. When extended to multiple timesteps, our approach consistently outperforms existing methods, showing substantial improvements across eight continuous control tasks from OpenAI Gym.
Rong Xiao, Zhiyuan Hu, Ping He et al.· IEEE Transactions on Image P...· 0 citations
