SpeechDETECT: an explainable automated speech processing pipeline for early detection of neurological and health changes
Abstract
Early detection of cognitive impairment remains a critical public health challenge. While biomarkers such as neuroimaging and cerebrospinal fluid analyses offer high sensitivity, their limited accessibility hampers widespread screening, especially in underserved settings. Speech-based markers have emerged as promising, noninvasive indicators of cognitive decline. To develop and validate SpeechDETECT, an end-to-end speech-processing pipeline that captures fine-grained acoustic and temporal markers of cognitive impairment and provides interpretable outputs suitable for large-scale screening. SpeechDETECT comprises six modules: (1) noise reduction / amplitude normalization; (2) an eight-domain voice-analysis framework (e.g., frequency parameters, speech fluency); (3) 50 ms segment-level feature extraction; (4) feature visualization; (5) dimensionality reduction / selection (Joint Mutual Information Maximization, LassoNet, PCA); and (6) classifier training with SHapley Additive exPlanations (SHAP). Performance was benchmarked against six acoustic toolkits (e.g., GeMAPS) on two English datasets: the DementiaBank Pitt corpus (train = 166, test = 71) with single cookie-theft picture description task and NIA PREPARE Phase 2 corpus (train = 1 064, test = 267) with multiple speech tasks. A Multi-Layer Perceptron trained on PCA-derived SpeechDETECT features achieved an F1-score = 0.81% and AUC-ROC = 0.80 on the Pitt test set, outperforming the best competing toolkit (AUC = 0.76). On the PREPARE test set—comprising ≤ 30 s recordings from four speech tasks—the same model attained F1 ≈ 0.67% and AUC-ROC = 0.70, demonstrating good generalizability. Cumulative-gains analysis showed that screening the top 40% of ranked participants captured ~ 70% of cognitively-impaired (CI) cases in Pitt and ~ 63% in PREPARE. SHAP revealed speech-fluency metrics (hesitation rate, pause ratio) and high-frequency formant dynamics as the most discriminative features. SpeechDETECT delivers accurate (AUC up to 0.80) and interpretable detection of early cognitive impairment across both structured and multi-task speech settings. Its fully automated, domain-informed approach enables scalable, speech-based screening and provides a foundation for multimodal systems that combine acoustic markers with clinical or biomarker data to further improve diagnostic precision. The SpeechDETECT toolkit is openly available on GitHub at https://github.com/SpeechCARE/SpeechDETECT-Toolkit for researchers and clinicians. A demo tutorial video showing pipeline usage is available at https://github.com/SpeechCARE/SpeechDETECT-Toolkit/blob/main/SpeechDETECT.mp4.