High-Fidelity Speech Reconstruction Architectures Utilizing Spectral Attention and Temporal Memory Units to Enhance Robust Signal Quality

Abstract

This paper introduces a high-fidelity speech reconstruction system that enhances speech quality and robustness even in challenging environments. The introduced model comprises a Spectral Attention and a Temporal Memory Unit that improve spectral and temporal representations of the speech signal, respectively. The Spectral Attention exploits multihead attention to concentrate on critical frequency components of the input by considering features of the STFT-based speech signal. In addition, the Temporal Memory Unit leverages a bidirectional LSTM to capture long-range temporal dependencies and maintain the naturalness of the output speech signal. The proposed model was tested under various acoustic environments, and its performance showed considerable improvements compared to the traditional speech reconstruction techniques. For example, the introduced model improved the SNR score from 10.2 dB to 24.8 dB, the PESQ score from 2.4 to 4.1, and the MOS score from 2.6 to 4.3. Furthermore, additional robustness analysis conducted under reverberant and packet loss conditions confirmed the stability of the introduced model. Consequently, the introduced model can be applied to real-time communications, hearing assistive devices, and speech-based intelligent systems.