C-MRL: Learning Directed Cross-Modal Relational Graphs for Multimodal Brain Network Analysis
Abstract
Multimodal brain networks integrate complementary structural and functional information to enhance the sensitivity of disease-specific features, thereby improving early diagnosis performance. However, existing methods typically model brain networks with predefined structures, which fail to capture task-specific interaction patterns, thereby limiting diagnostic performance and model explainability. To address this, we propose a Cross-Modal Relational Graph Learning (C-MRL) framework that transforms cross-modal connections into a learnable objective and employs a specially designed GNN learner to achieve bidirectional cross-modal interaction, enabling the construction of task-adaptive brain networks. C-MRL not only achieves superior performance but also provides new insights into the synergy of multimodal data. Specifically, C-MRL introduces two main designs: first, it constructs a Learnable Cross-modal Relational Graph (LCRG) based on neuroanatomical priors, where bidirectional cross-modal interactions are realized through two complementary directed acyclic graphs, each capturing unidirectional nonlinear relationships between brain regions. Second, based on the learned directed graph, we develop an Attentive Hierarchical Multimodal Fusion Learner (AHFL) that performs bidirectional message propagation and aggregation, outputting task-discriminating graph representations that integrate subject-specific multimodal information. Extensive experiments on four benchmark datasets demonstrate the superiority of C-MRL. In particular, it achieves state-of-the-art performance on brain disease classification, with the full model reaching 92.5%±1.8 accuracy for Alzheimer's disease and exhibiting robust cross-dataset generalization (91.4%±1.1 accuracy on the unseen AIBL dataset), substantially improving over existing methods. Our code is available at https://github.com/LiuFei-AHU/CMRL.