A CLOSE-SET MULTI-SPEAKER IDENTIFICATION via LIGHTWEIGHT CNN ARCHITECTURES LEVERAGING MFCC-DERIVED SPECTRAL FEATURES

Abstract

Speaker identification is an important task in speech processing, with applications in security, authentication, forensics, and personalised human–computer interaction. This study presents a lightweight convolutional neural network (CNN)-based system for close-set identification of 40 speakers using Mel-Frequency Cepstral Coefficients (MFCCs). A dataset comprising 1,000 speech samples was collected from 40 speakers, with 850 samples used for training and 150 for validation. MFCC features were extracted from the speech signals and used as input to a compact CNN consisting of four convolutional blocks, three intermediate pooling operations, adaptive average pooling, and a fully connected classification layer. The proposed architecture contains 102,792 trainable parameters. Under the evaluated controlled conditions, the model achieved 100% validation accuracy, with all 150 validation samples correctly classified, and achieved an ROC-AUC of 1.00 for all 40 speaker classes. Pooling ablation experiments showed that removing any one of the three intermediate pooling operations reduced the best validation accuracy from 100.00% to 99.33%, supporting the use of the complete pooling configuration. An augmentation sensitivity analysis further showed that the unaugmented configuration achieved the highest validation accuracy of 100.00%, while pitch-shift augmentation resulted in 99.00% and Gaussian-noise augmentation reduced accuracy to 2.67%, both independently and when combined with pitch shifting. These findings indicate that the unaugmented configuration was the most effective setting for the proposed model under the evaluated conditions. Overall, the proposed lightweight CNN demonstrates effective closed-set speaker identification under controlled recording conditions, while further evaluation is required for open-set recognition, speaker verification, and greater recording variability.