An empirical comparison of deep CNN architectures for heart sound classification from phonocardiogram signals
TL;DR
This work compares five deep convolutional neural networks, namely LeNet5, AlexNet, VGG16, ResNet-50 and a hybrid LSTM-CNN and a hybrid LSTM-CNN, on the task of separating normal from abnormal heart sounds, and reports accuracy, AUC, sensitivity, specificity, F1 score and the Matthews correlation coefficient.
Abstract
Heart disease continues to be the world’s number one killer, making low-cost, early screening a priority. A phonocardiogram (PCG), a simple recording of sounds made by the heart, provides valuable clues to conditions such as murmurs and valve defects and can be recorded cheaply using a digital stethoscope. With less accessibility to specialists, the automatic analysis of heart sounds is preferred for screening and tele-health environments. We compare five deep convolutional neural networks in this work, namely LeNet5, AlexNet, VGG16, ResNet-50 and a hybrid LSTM-CNN, on the task of separating normal from abnormal heart sounds. Before entering the network, each recording is converted to a log-Mel spectrogram, and each model is trained at five different learning rates (1e-2, 1e-3, 5e-4, 1e-4 and 5e-5). The PhysioNet/CinC Challenge 2016 database is common to all experiments and, after duplicate removal, provides 3, 240 recordings. A leakage-free protocol is enforced throughout: exact duplicates are detected by audio content hashing and the split is patient-independent, which on this dataset is sufficient to prevent several networks from reaching a misleading 100% accuracy. We report accuracy, AUC, sensitivity, specificity, F1 score and the Matthews correlation coefficient. Under five-fold cross-validation VGG16 attains the highest mean accuracy (91.94% ± 1.59, AUC 0.968, MCC 0.745), marginally ahead of ResNet-50 (90.93% ± 1.09); McNemar’s test confirms that the difference between the leading architectures is not statistically significant (p = 0.31), so the two are reported as jointly strongest. An ablation study shows that spectrogram augmentation, rather than ImageNet pre-training, is the decisive design choice, and every model fails at a learning rate of 1e-2. External validation on the CirCor DigiScope 2022 database shows that zero-shot cross-dataset transfer performs at chance level, while the retrained pipeline exceeds the majority baseline, underlining how strongly single-dataset results can overstate real-world generalisation.