ACES: ascertaining diagnosis classification with elicited speech in individuals with heterogeneous, comorbid psychiatric disorders.
Automated analysis of speech and language provides a critical opportunity for developing a scalable tool for general medical settings to aid in psychiatric diagnosis and triage. In a cross-diagnostic highly comorbid sample, we evaluated the contribution of speech and language features to diagnostic classification through a hierarchical, dichotomous approach: first distinguishing healthy volunteers (HV) vs. participants with any psychiatric disorder (Split 1), then identifying those with serious mental illness (SMI) vs. other psychiatric disorders (Split 2). Speech was collected from 266 participants via picture description, verbal fluency, paragraph reading, and open-ended verbal journaling tasks. We extracted 640 interpretable features spanning acoustic, temporal, lexical, syntactic, discourse, and coherence domains. LightGBM models were trained with 5-fold cross-validation comparing all combinations of speech tasks, and SHAP values were plotted for feature importance. For Split 1, the best-performing model achieved F1=0.865 combining picture description and journaling tasks, with picture description alone reaching F1=0.830. For Split 2, performance was moderate (F1=0.626), with picture description also as the best single task. Top features for Split 1 included amplitude instability (shimmer) and restricted pitch variance; Split 2 was influenced by articulation rate and filled pauses. This study represents a novel classification approach in a naturalistic, clinically complex sample. Findings suggest that a brief, explainable speech-based assessment may be able to identify individuals who need further evaluation for psychiatric disorders. External validation, bias auditing, and deployment studies are warranted to assess clinical impact.