Interpretation Before Integration: LLM-Guided Multimodal Completion and Fusion Network for Survival Analysis With Incomplete Data

Abstract

Multimodality survival analysis for nasopharyngeal carcinoma (NPC) holds great potential for improving prognosis prediction and clinical decision-making. However, it is challenged by structural and semantic misalignments across heterogeneous data. Structural misalignment arises from incomplete clinical records, where missing data introduce uncertainty in prediction. Semantic misalignment stems from the gap between structured modalities (e.g., clinical and radiomic features) and unstructured data such as 3-D magnetic resonance imaging (MRI), hindering effective feature integration. Existing methods often ignore missing data or compress multimodal information into scalar representations, failing to capture complex modality interactions and solve the problem of semantic misalignment. Furthermore, current completion techniques typically lack interpretability and overlook joint modeling of inter- and intra-sample correlations when dealing with structural misalignment, limiting their reliability in clinical settings. These issues are further exacerbated by over-parameterized models prone to overfitting in small-sample scenarios. To address these challenges, we propose LMCF, a large language model guided multimodal completion and fusion (LMCF) network tailored for survival analysis with incomplete data. LMCF consists of two core components: a lightweight dual-branch multimodality enhanced feature encoding (LDME) layer, which incorporates an interpretable multisource cross-modality completer (IMCC) for explainable reconstruction of missing data to resolve structural misalignment; and a large language model (LLM)-guided structure-semantic two-stream fusion (LSTF) layer, equipped with a quaternion convolution-based cross-domain adaptive attention fusioner (QCAAF) to effectively integrate features across modalities and mitigate semantic misalignment. Extensive experiments on the Cancer Genome Atlas (TCGA) and two proprietary NPC datasets [postradiation nasopharyngeal necrosis (PRNN) and nasopharyngeal carcinoma dataset (NCD)] from Sun Yat-sen University Cancer Center demonstrate LMCF’s superior performance in survival prediction and risk stratification, particularly under conditions of incomplete modalities and limited data resources.