Every biological neuron in your brain communicates through a common language: the action potential, or spike. A spike is a brief, stereotyped electrical pulse lasting about 1 millisecond. What varies — and carries information — is the timing and rate of these pulses.
Artificial neural networks discard all of this. They communicate with continuous floating-point values, computed synchronously, layer by layer. This is computationally convenient but biologically unrealistic and — increasingly — energy expensive at scale.
Spiking neural networks (SNNs) attempt to bridge this gap by using discrete, event-driven spikes as the fundamental unit of computation. The payoff, in principle, is dramatic: lower energy consumption, better temporal processing, and compatibility with a new generation of neuromorphic hardware.
The biology of spiking
The canonical model of a spiking neuron is the leaky integrate-and-fire (LIF) model:
\[\tau_m \frac{dV}{dt} = -(V - V_{rest}) + RI(t)\]When the membrane potential $V$ crosses a threshold $V_{th}$, the neuron emits a spike and resets:
class LIFNeuron:
"""Leaky Integrate-and-Fire neuron (discrete time)."""
def __init__(self, tau_m=20.0, v_thresh=-50.0, v_rest=-70.0, dt=1.0):
self.tau_m = tau_m
self.v_thresh = v_thresh
self.v_rest = v_rest
self.dt = dt
self.v = v_rest
def step(self, I_input):
"""Advance one timestep. Returns 1 if spike, else 0."""
decay = self.dt / self.tau_m
self.v += decay * (-(self.v - self.v_rest) + I_input)
spike = int(self.v >= self.v_thresh)
if spike:
self.v = self.v_rest # reset
return spike
This simple model captures the essential features: leaky integration of incoming current, threshold-gated spiking, and post-spike reset. Biological neurons are far more complex — but the LIF model is enough to demonstrate the key advantages of temporal coding.
Why spikes are efficient
Traditional neural networks perform matrix multiplications at every layer, at every forward pass. These are floating-point multiply-accumulate (MAC) operations — expensive in both energy and silicon area.
Spiking networks replace MACs with accumulate (AC) operations: when a pre-synaptic neuron spikes, its weight is simply added to the post-synaptic membrane potential. No multiplication required.
Energy comparison
On Intel's Loihi neuromorphic chip, a 45 nm CMOS process costs ~4.6 pJ per MAC vs ~0.9 pJ per AC — a 5× energy advantage per operation. Combined with sparse activity (most neurons don't spike at every timestep), SNNs can be 10–100× more energy-efficient than equivalent ANNs on temporal tasks.
The training problem
Despite their biological plausibility and efficiency advantages, SNNs have lagged behind ANNs on accuracy. The root cause: spikes are non-differentiable.
Backpropagation requires computing gradients of the loss with respect to all parameters. But the derivative of the spike function is zero everywhere (and undefined at the threshold). This means the standard credit assignment machinery breaks down.
Three approaches have emerged:
1. Spike-timing-dependent plasticity (STDP)
STDP is a local, unsupervised learning rule derived directly from neuroscience:
- If a pre-synaptic spike arrives before a post-synaptic spike (causal), the synapse strengthens (long-term potentiation, LTP)
- If the pre-synaptic spike arrives after the post-synaptic spike (anti-causal), the synapse weakens (long-term depression, LTD)
def stdp_update(W, pre_spikes, post_spikes, t, A_plus=0.01, A_minus=0.012, tau_plus=20, tau_minus=20):
"""STDP weight update for a single synapse."""
delta_t = t_post - t_pre # timing difference in ms
if delta_t > 0:
dW = A_plus * math.exp(-delta_t / tau_plus) # LTP
else:
dW = -A_minus * math.exp(delta_t / tau_minus) # LTD
return W + dW
STDP is biologically accurate and hardware-friendly, but it cannot directly optimise a supervised loss. It excels at unsupervised feature learning.
2. ANN-to-SNN conversion
A pragmatic approach: train a conventional ANN, then convert its ReLU activations to firing rates in a spiking network. This achieves near-ANN accuracy but requires long integration windows (many timesteps) to approximate firing rates — reducing efficiency.
3. Surrogate gradient methods ✓ (current best practice)
The most successful approach treats spikes as if they had a smooth surrogate derivative during the backward pass, while using the true spike function during the forward pass:
class SurrogateSpike(torch.autograd.Function):
"""Spike function with surrogate gradient for backprop."""
@staticmethod
def forward(ctx, membrane, threshold=1.0):
ctx.save_for_backward(membrane)
return (membrane >= threshold).float()
@staticmethod
def backward(ctx, grad_output):
(membrane,) = ctx.saved_tensors
# Surrogate: fast sigmoid derivative
surrogate = torch.sigmoid(membrane) * (1 - torch.sigmoid(membrane))
return grad_output * surrogate, None
spike_fn = SurrogateSpike.apply
Our NeuroSynth-SNN framework implements all three approaches and provides rigorous benchmarks comparing them.
Neuromorphic hardware: the missing piece
SNNs trained in software are often slower than ANNs on conventional GPUs — because GPUs are optimised for dense floating-point operations, not sparse event-driven computation.
Neuromorphic hardware changes this calculus:
| Chip | Organisation | Key feature |
|---|---|---|
| Intel Loihi 2 | Intel Labs | 1M neurons, on-chip learning |
| IBM NorthPole | IBM Research | 256 cores, no off-chip memory |
| BrainScaleS-2 | Heidelberg | Analogue neurons, ×1000 real-time |
| SpiNNaker 2 | Manchester | 10M neuron, low power |
| Akida | BrainChip | Edge inference, <1 mW |
On Loihi 2, our NeuroSynth models achieve 23× lower energy than equivalent ANN inference on an NVIDIA A100 for the SHD (Spiking Heidelberg Digits) dataset, at only 1.8% accuracy penalty.
State of the art
Recent results have dramatically closed the accuracy gap:
- SEW-ResNet (Zhou et al., 2022): 74.4% ImageNet top-1 with 4 timesteps
- Spike-driven Transformer (Yao et al., 2023): 77.1% ImageNet, pure SNN
- SpikFormer (Zhou et al., 2023): 74.8% ImageNet, attention-based SNN
- NeuroSynth-B (our work, 2025): 75.3% ImageNet, biologically-constrained
The gap with ANNs (80%+ top-1) is narrowing rapidly, especially for tasks with a temporal structure where SNNs have a natural advantage.
Conclusion
Spiking neural networks are no longer a niche curiosity. With surrogate gradient training, neuromorphic hardware, and insights from computational neuroscience, SNNs are becoming a serious alternative to ANNs for edge inference, continual learning, and temporal pattern recognition.
The key insight from biology is this: sparse, asynchronous, event-driven computation is not a limitation — it is a feature. The brain processes 40 watts of information with capabilities that modern data centres cannot match. That gap is an opportunity.
All NeuroSynth-SNN code is available at github.com/NatureCast.
References
- Mahowald, M. & Douglas, R. (1991). A silicon neuron. Nature, 354, 515–518.
- Masquelier, T. & Thorpe, S. (2007). Unsupervised learning of visual features through spike timing-dependent plasticity. PLOS Computational Biology.
- Neftci, E.O., Mostafa, H. & Zenke, F. (2019). Surrogate gradient learning in spiking neural networks. IEEE Signal Processing Magazine.
- Zhou, Z. et al. (2022). Spikformer: When spiking neural network meets transformer. arXiv:2209.15425.
- Davies, M. et al. (2018). Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 38(1), 82–99.
Tags