Enhancing multimodal survival prediction: tri-modal learning with clinical knowledge integration via state space models

Abstract

Accurate survival prediction is crucial for precision oncology, yet it faces challenges due to the neglect of clinical priors and high computational complexity. We propose TriBind-Mamba, a tri-modal framework integrating Clinical Knowledge Prompting (CKP) and selective State Space Models (SSMs). By transforming structured clinical records into semantic narratives using Large Language Models (LLMs), our model provides high-level context for morphological and molecular features. TriBind-Mamba efficiently processes gigapixel whole slide images and transcriptomic profiles with linear complexity, achieving state-ofthe-art performance (Overall C-index of 0.664) across five TCGA cohorts while significantly reducing computational overhead. Interpretability is enhanced by integrating human-readable clinical knowledge prompts, biologically meaningful pathway-level transcriptomic tokens, and WSI attention heatmaps that project model-derived importance scores back onto histopathological regions. These analyses suggest that TriBind-Mamba focuses on prognostically relevant malignant areas, providing a more transparent basis for multimodal survival prediction.