Multi-head attention-driven multimodal feature integration network for autism spectrum disorder detection

Abstract

The growing prevalence of Autism Spectrum Disorder (ASD) highlights the need for accurate and reliable intelligent screening systems for early behavioral assessment. However, ASD-related behaviors vary significantly across individuals, making diagnosis based on a single modality or subjective evaluation unreliable. Moreover, existing computational approaches often struggle to effectively model complex spatial and temporal dependencies in behavioral video data, leading to limited feature representation. To address these challenges, this study proposes a Multi-head Attention-driven Multimodal Feature Integration Network (MAtMFIN) that jointly analyzes Eye-Tracking (ET) scanpath data and behavioral video data to improve the robustness of ASD screening. The framework employs a Hierarchical Attention Refinement block (HARb) and a Spatial Enhancement Module (SEM) for effective feature refinement, along with a cross-modal cross-attention mechanism to capture complementary relationships between modalities. Experimental evaluation on the proposed multimodal ASD dataset demonstrates that the proposed MAtMFIN framework consistently outperforms state-of-the-art transfer learning and Deep Learning (DL) models, including a hybrid Convolutional Neural Network (CNN)-Long Short-Term Memory (LSTM) model, 2D-CNN with attention, LSTM-attention, Inflated 3D CNN, and Spatio-Temporal Graph Neural Networks (STGNN). The proposed method achieves a training accuracy of 94.6%, a testing accuracy of 93.2%, and an F1 score of 93.5%. These results indicate that effective cross-modal feature integration and attention-based modelling of spatio-temporal dependencies significantly enhance the ASD behavioral analysis, highlighting the potential of the proposed framework for intelligent healthcare applications.