TL;DR
SymbolicLight V1 is presented, a spike-gated dual-path language model that couples binary Leaky Integrate-and-Fire dynamics with a continuous residual stream that shows no clear accuracy separation across five zero-shot benchmarks.
This work constructs a fully TTFS-based SNN architecture and train it end-to-end, and introduces a reference-based strategy specifically to encode the four core LLM components: embedding layers, layer normalization, attention-related operations and dropout.
Large language models (LLMs) achieve strong performance across many tasks but rely on dense multiply-accumulate (MAC) operations during inference, resulting in high energy cost. Spiking neural networks (SNNs) offer an event-driven alternative in which synaptic integration uses lightweight accumulation. However, spike-d...
The 194M-parameter model is implemented on an Alveo U50C FPGA using digital fixed-point arithmetic and on an ARM CPU using sparse integer execution to connect event sparsity to omitted computation and data movement in SymbolicLight V2.
The BabyLM challenge measures how much language a model can learn from developmentally-plausible, child-scale data rather than internet-scale corpora, yet prior language models forgo the biological constraints of the neural circuitry that acquires human language: spiking neurons separated into excitatory and inhibitory...
SpikeWorld, a 1.45M-parameter sparse spiking model jointly trained for heterogeneous sensory prediction, semantics, image-text binding and action-conditioned dynamics, is introduced, showing that the contribution is not superior linear identification, but its integration with a frozen multimodal spiking checkpoint.
Lapis is proposed, a spiking attention mechanism that scores each token pair by the L1 distance between its query and key first-spike latency vectors under time-to-first-spike coding, and maps this distance to an affinity through a Laplacian kernel.
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.
