All-optical speech recognition based on nonlinear diffractive neural network with picosecond latency

Abstract

We propose and experimentally validate a keyword wake-up approach based on an all-optical nonlinear diffractive deep neural network (AON-D2NN) chip. The chip monolithically integrates an input layer, nine hidden layers, and an output layer. The hidden layers employ thermal phase shifter arrays for programmable matrix operations, and germanium microstructures provide on-chip nonlinear activation functions. We preprocess speech signals via fast Fourier transform (FFT) and intensity sampling at characteristic frequencies, then encode them into voltage vectors for parallel input. Using the stochastic gradient descent algorithm for in-situ training in the optical domain, we achieve a recognition accuracy of 75% in a binary classification task with the AON-D2NN chip. Benefiting from the all-optical forward propagation architecture, the chip avoids frequent photoelectric conversions and reduces its computation latency to 172 ps. This work offers a viable solution for low-power and low-latency optical-domain speech recognition.