Multimodal Anxiety Detection with Adaptive Hyperbolic Few-Shot Learning

Abstract

Anxiety disorders impact millions globally, yet traditional diagnosis relies on clinical interviews, while machine learning models struggle with limited labeled data. In this work, we introduce the Hyperbolic Curvature Few-Shot Learning Network (HCFSLN), a novel framework for multimodal anxiety detection using speech, physiological signals, and video. Our approach leverages hyperbolic embeddings to better capture hierarchical structure in affective signals, along with cross-modal attention and adaptive gating to model interactions across modalities. We collect a multimodal dataset from 108 participants and evaluate HCFSLN under few-shot settings. Our results show that HCFSLN significantly improves robustness and generalization, achieving 88% accuracy and outperforming baselines by 14%. Audio achieves the best unimodal performance (0.88 on social anxiety dataset, 0.79 on our dataset in 1-shot); adding physiological signals improves accuracy by ∼ 3–5%, and tri-modal fusion reaches up to 0.86 (social anxiety dataset) and 0.83 (our dataset), while using all modalities slightly reduces performance (∼ 0.79, ∼ 0.75) due to noise. The adaptive gating mechanism provides interpretability by weighting modalities, consistently prioritizing audio and leveraging physiological signals, with t-SNE showing clearer class separation. Further analysis highlights complementary multimodal cues for anxiety detection. Our approach is effective in low-data settings. We release our dataset and code at https://huggingface.co/datasets/SIR-Lab/Multi-Modal-Anxiety-Dataset.