Automated depression detection from human speech using acoustic feature analysis and deep learning

Abstract

Depression is a prevalent mental health condition impacting millions worldwide; however, many cases remain undiagnosed due to social stigma, limited clinical accessibility, and reliance on subjective assessment methods. Automated speech analysis offers a promising, non-invasive approach for early diagnosis by identifying behavioral and physiological patterns expressed in vocal features. This study proposes a novel multimodal depression recognition framework that combines handcrafted acoustic characteristics with deep learning-based representations of Mel spectrograms. The primary objective is to classify speech samples into three distinct psychological conditions: normal, mild depression (Stage 1), and severe depression (Stage 2). Deep embeddings are extracted using a Convolutional Neural Network (CNN) and fused with traditional acoustic descriptors, including energy, pitch, loudness, Mel-Frequency Cepstral Coefficients (MFCCs), and spectral attributes. The fused feature representation is evaluated across ensemble and classical machine learning classifiers. Comparative results demonstrate that the multimodal fusion approach significantly outperforms single-modality baselines; the fusion model achieves 99.25% accuracy using Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), and k-Nearest Neighbors (kNN) classifiers, whereas the Random Forest baseline achieved 31% on fused features and 91% on acoustic features alone. Visualization techniques further enhance model interpretability by uncovering discriminative acoustic and spectral patterns related to depression. These findings highlight the potential of AI-based speech recognition as a scalable and objective preliminary mental health screening tool.