INRC Forum: James Knight

Efficient training of sparse SNN classifiers using GeNN

Abstract:Intuitive and easy to use application programming interfaces such as Keras have played a large part in the rapid acceleration of ANN-based machine learning. We want to unlock the potential of spike-based machine learning in the same way, so here we present mlGeNN, an easy way to define, train and test spiking neural networks using GeNN — our efficient GPU-accelerated SNN simulator. Using GeNN, we demonstrate that we can use e-prop to train recurrent SNN classifiers on datasets including the Spiking Heidelberg Digits (SHD) and DVS gesture. We show that these classifiers can not only offer comparable performance to LSTMs but are up to 7× faster when performing inference on the same GPU hardware. As GeNN is designed to exploit sparse connectivity, by replacing the dense recurrent connectivity in classifier models with random sparse connectivity, we can reduce the time taken to train such models by almost 10× — although this results in some reduction in classification accuracy. However, in biological brains, alongside the changes to the strength of existing synapses driven by synaptic plasticity, structural plasticity prunes unused synapses and forms new ones. The Deep-R learning rule provides a framework for combining gradient-based learning with structural plasticity and by combining Deep-R with e-prop, we demonstrate that the aforementioned reduction in classification accuracy can be eliminated, even in very sparsely connected models.

Bio: Jamie Knight received his BEng degree in Electronic Engineering from the University of Warwick in 2006. After working as a games developer for several years, he received an MPhil in Advanced Computer Science from the University of Cambridge in 2013 and a PhD in Computer Science from the University of Manchester in 2016. His doctoral work focussed on using the SpiNNaker neuromorphic supercomputer to simulate large-scale computational neuroscience models with synaptic plasticity. Since 2017 Jamie has worked at the University of Sussex, first as a Research Fellow focussing on using GPU hardware to accelerate spiking neural network based robot controllers and, since 2022, as a EPSRC Research Software Engineering Fellow focussing on spike-based machine learning and the software to enable it.

For the meeting link, see the full INRC Forum Spring 2023 Schedule (accessible only to INRC Affiliates and Fully Engaged Members).