Cross-Dataset Generalization in Phonocardiogram Murmur Classification: A Benchmark Study of Domain-Conditional Performance and the Role of Multi-Representation Fusion

Abstract

Background/Objectives: Automated phonocardiogram (PCG) analysis is a candidate tool for scaling up cardiac auscultation, but the internal accuracy reported in many studies may not reflect performance when a model is applied to recordings from a different source. This study evaluated how a broad range of classifiers generalize across PCG datasets and examined whether a multi-representation architecture improves cross-dataset transfer. Methods: Eleven approaches were trained on the pediatric CirCor DigiScope dataset and evaluated, without target-domain fine-tuning, on two external datasets: a heterogeneous adult/mixed dataset (the PhysioNet/Computing in Cardiology Challenge 2016 database) and a homogeneous five-class open-access heart sound dataset. The approaches spanned classical machine learning (spport vector machine, random forest), standard deep models (1D-CNN, 2D-CNN, ResNet1D, CNN-BiLSTM, and two hybrid networks), image-based backbones (EfficientNet-B0, MaxViT), and a proposed four-branch fusion model (PCG-MambaConformer) combining raw-signal, mel-spectrogram, and continuous wavelet transform (CWT) representations. Threshold-transfer and precision–recall analyses, adversarial domain adaptation, an uncertainty-quantification benchmark, a disease-specific transfer analysis, multi-architecture ensembling, a branch ablation, a label-grouping sensitivity analysis, a reciprocal (reverse-direction) transfer experiment and explainability analyses (feature-space t-SNE with class-separability metrics and Grad-CAM saliency) were also performed. Performance was summarized using the area under the receiver operating characteristic curve (AUROC) with sensitivity and specificity. Results: Internal AUROC was similar across deep models (approximately 0.82–0.90) and largely independent of parameter count, whereas classical models were lower (0.67–0.69). On the heterogeneous external dataset, the highest AUROC was 0.738 and none of the nine deep models reached a sensitivity and a specificity both of at least 0.80 at any of the three thresholds examined. On the homogeneous external dataset, PCG-MambaConformer reached an AUROC of 0.985 (sensitivity 0.718, specificity 0.995 at the 0.5 threshold), whereas other models tended to collapse toward either always-normal or always-abnormal predictions. Adversarial adaptation improved performance on the homogeneous target but reduced it on the heterogeneous target. On the heterogeneous target, even an oracle threshold raised balanced accuracy by only 0.022 on average, whereas on the homogeneous target a source-derived threshold produced a balanced operating point only for PCG-MambaConformer (sensitivity 0.863, specificity 0.980). Models trained in the reverse direction (PhysioNet/CinC Challenge 2016 to CirCor) also transferred poorly (AUROC 0.612–0.677). Branch ablation showed that three of the four single-branch configurations collapsed to a one-class operating point and that the full fusion model achieved the highest AUROC, and a feature-space analysis showed that successful transfer was associated with the class-separability of the target domain (silhouette score highest for the homogeneous dataset) rather than with the proximity of dataset centroids. Conclusions: In the two external datasets evaluated, successful cross-dataset PCG transfer was associated with the homogeneity of the target domain; because these datasets also differed in population, recording protocol, disease spectrum, class distribution and signal quality, homogeneity cannot be isolated as the sole determinant of transferability. Internal accuracy was a weak indicator of external behavior, model complexity alone did not close the heterogeneous transfer gap, and balanced cross-dataset performance on a homogeneous target was associated with multi-representation fusion. The near-perfect result on the homogeneous target should be read as technical cross-dataset transfer under favorable, highly curated conditions rather than as clinical external validation, whereas the heterogeneous result is the more realistic indication of current limitations. These findings support multi-metric external evaluation on independent data, followed by prospective clinical validation, before any clinical use is considered.