Bidirectional Cross-Modal Attention with Adaptive Thresholding for Multimodal Depression Detection Across the EATD-Corpus and DAIC-WOZ Datasets

Abstract

Background: Early and accurate detection of depression from multimodal data is a critical yet challenging task. Many existing models either rely on a single modality or exhibit limited robustness across datasets, while publicly available datasets are often small and highly imbalanced. Methods: In this study, we propose a robust multimodal fusion framework that leverages bidirectional cross-modal attention to effectively integrate audio and text features, allowing the model to learn complementary information from both modalities. To address class imbalance and limited data, we employ SMOTE-based oversampling. Our model further incorporates validation-based adaptive thresholding and probability ensembling to enhance decision robustness and generalization. Experiments are conducted on two benchmark datasets, EATD-Corpus (Chinese; 162 participants) and DAIC-WOZ (English; 189 usable interviews), and ablation experiments compare cross-modal attention settings. Results: On EATD-Corpus, our fusion model achieves strong performance (F1: 0.820, Recall: 0.934, Precision: 0.762), achieving higher mean performance than the text-only and audio-only baselines evaluated under the same participant-level five-fold protocol. On DAIC-WOZ, the framework remains competitive (F1: 0.836, Recall: 0.925, Precision: 0.775). The ablation results confirm that bidirectional cross-modal interaction yields the most reliable trade-off between Precision and Recall. Conclusions: This paper proposes a deep learning framework for depression screening that jointly leverages audio and text modalities, demonstrating robust performance on small, class-imbalanced datasets and architectural generalizability across two corpora with different languages and interview settings.