Stable and compute-resource efficient learning with spiking and quantum neural networks

Abstract

The second-generation neural networks have evolved in recent years, which have become more complex architectures such as spiking neural networks and quantum neural networks. However, the computational resource restriction of neural networks on edge devices is still challenging. The thesis investigates stable learning and compute-resource efficiency on spiking neural networks and hybrid classical-quantum neural networks. Other common qualities like high performance (e.g. high accuracy, high reward), robustness, convergence, predictability, and fast running times were also considered in one or more studies. The contributions of the thesis have several folds. The first study was using audio data; one reason was to verify if a trend called temporal information concentration is present in the spiking neural network. I also gathered other findings, such as dataset complexity impacting Fisher information, related to temporal information dynamics. A second study on spiking neural networks revealed that temporal information concentration was not present in quantization aware training variants, but an increase in Fisher information was found in those variants. The third study on my Multimodal Simplified Spiking Neural Networks explored the effects of audio and image noise. The results show the multimodal model outperformed its unimodal counterparts, but certain configurations of image noises, audio noises, and noise levels performed better than others. In one of the hybrid classical-quantum neural network studies with reinforcement learning, in my quantization-aware training variant, I found higher initial reward growth, longer decreasing in standard deviation and policy entropy, and a few correlations as well related to average reward and policy entropy. In the second study on hybrid classical-quantum neural networks, structured pruning is found to sharpen decisiveness and reveal bad pruning paths, while overparameterization can help exploration. All these studies try to address maintaining or improving stable learning and if the models are computation-resource efficient enough to be realistic.