MultiSigBERT: Beyond Survival Analysis through Multimodal and Sequential Modeling in Oncology

Enhancing multimodal survival prediction: tri-modal learning with clinical knowledge integration via state space models

Accurate survival prediction is crucial for precision oncology, yet it faces challenges due to the neglect of clinical priors and high computational complexity. We propose TriBind-Mamba, a tri-modal framework integrating Clinical Knowledge Prompting (CKP) and selective State Space Models (SSMs). By transforming structured clinical records into semantic narratives using Large Language Models (LLMs), our model provides high-level context for morphological and molecular features. TriBind-Mamba efficiently processes gigapixel whole slide images and transcriptomic profiles with linear complexity, achieving state-ofthe-art performance (Overall C-index of 0.664) across five TCGA cohorts while significantly reducing computational overhead. Interpretability is enhanced by integrating human-readable clinical knowledge prompts, biologically meaningful pathway-level transcriptomic tokens, and WSI attention heatmaps that project model-derived importance scores back onto histopathological regions. These analyses suggest that TriBind-Mamba focuses on prognostically relevant malignant areas, providing a more transparent basis for multimodal survival prediction.

From data silos to integrated diagnosis: multimodal machine learning in glioblastoma

Effective glioblastoma care requires integrating multiparametric longitudinal MRI with histopathology, molecular profiling, and clinical records documenting surgery, radiotherapy, chemotherapy, and supportive treatments. In routine practice, however, these data streams are often evaluated separately rather than jointly, which can delay molecularly informed stratification, limit reproducibility across centers, and complicate interpretation of post-treatment imaging changes. Multimodal machine learning (MML) provides a framework for clinical decision support by integrating diverse patient data across the course of care, from symptom presentation through diagnosis to treatment decisions. By combining MRI, whole-slide pathology, molecular and methylation profiling, and treatment timelines derived from electronic health records, MML models can capture disease characteristics over time across biological scales through representation learning and multimodal fusion. Importantly, these approaches can incorporate uncertainty through model calibration and confidence-aware predictions. When rigorously developed and validated, MML models may generate clinically relevant outputs, including integrated diagnosis, molecular classification, individualized survival estimates, probabilistic discrimination between tumor progression and pseudo-progression, and stratification for clinical trial eligibility. In this Mini Review, we summarize recent advances and emerging translational evidence for clinically oriented MML in glioblastoma, with particular emphasis on MRI-centered systems that integrate imaging with pathology, selected molecular measurements, and longitudinal clinical context. We also outline key methodological and practical considerations, including dataset curation, leakage control, external validation, calibration, and post-deployment monitoring—required to support safe, robust, and generalizable implementation of MML approaches in neuro-oncology practice.

Enhancing stratification for survival analyses across standardized data sources

Patient stratification is crucial for advancing personalized medicine yet is complicated due to the fragmented and variable nature of healthcare data. The Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) addresses the challenges regarding the data by offering a standardized framework for data integration across diverse sources and configurations. Recent advancements in machine learning, particularly transformer-based models such as Bidirectional Encoder Representations from Transformers (BERT), have demonstrated significant potential in extracting deep patient representations from electronic health records. This study assesses the efficacy of BERT-based patient representation learning using OMOP CDM data for patient stratification. We harmonize originally incompatible datasets, including the established MIMIC-IV-2.2 and lung cancer data from the German cancer registry Schleswig-Holstein, within the OMOP CDM framework. BERT is pre-trained on the MIMIC-IV-2.2 dataset, and the derived representations are utilized to generate patient embeddings from the cancer registry (test) data. We employ k-means clustering on the embeddings to stratify patient subgroups. To evaluate whether the embeddings are useful for clustering, we divide the original cancer registry data into the corresponding groups and conduct survival analyses on selected columns for each cluster. The clustering method’s effectiveness is assessed by comparing survival models trained on these clusters with naïve clusters derived only from the original dataset. In addition, we included a clinical expert review, in which a physician assessed the resulting cluster assignments for clinical plausibility and interpretability. Our approach effectively identifies patient similarities across datasets and allowed for efficient patient stratification. Survival analyses show varied performance depending on the model and cluster characteristics, with up to a 11% improvement over naïve k-means clusters, demonstrating the benefits of transfer learning. For some groups of patients, the corresponding accuracy of the survival analysis increased by up to 7%, emphasizing the value of stratifying homogeneous subgroups. Utilizing standardized data and transformer-based foundation models to generate patient embeddings demonstrates effective knowledge transfer between two vastly different datasets and enables the identification of groups in which the accuracy of survival analysis can be significantly increased.

A Markov Blanket-based framework for dependency-aware feature selection in multimodal breast cancer data

Explainable AI for analyzing cancer outcomes using large-scale genome sequencing data

Metastatic cancer remains a leading cause of global mortality, yet accurate prognosis is frequently hampered by high-dimensional molecular features and heterogeneous clinical presentations. While traditional staging systems and linear models provide a foundational risk assessment, they often fail to capture the complex, nonlinear interactions between metastatic topology, genomic burden, and functional sequence variation. To address this, recent advances in machine learning and genomic foundation models present a transformative opportunity to integrate diverse data types into an explainable predictive framework. Consequently, this research developed a multi-tier, explainable AI framework designed to risk-stratify patients and predict overall survival using clinical and genomic covariates. Additionally, the framework aimed to surface sequence-level disease drivers by implementing joint variant calling from RNA-seq data and leveraging transformer-based architectures. The study employed a two-track methodological approach encompassing populationscale modeling and sequence-level deep learning. For the population-scale aim, a retrospective analysis was conducted on the Memorial Sloan Kettering-Metastatic cohort, consisting of 25,775 patients. Five distinct classifiers XGBoost, Logistic Regression, Random Forest, Decision Tree, and Naive Bayes were trained on a balanced subset of 20,338 patients utilizing an 80/20 stratified split. Model explainability was established through Shapley Additive Explanations (SHAP), while survival dynamics were evaluated using Kaplan-Meier estimates, Cox proportional hazards models, and an XGBoost-Cox variant. Concurrently, a pilot study involving 60 individuals, comprising 30 breast cancer cases and 30 controls, investigated sequence-level drivers using RNAseq data. A joint variant calling pipeline generated a unified genomic variant call format for association testing, and three genomic foundation models DNABERT-2, HyenaDNA, and Nucleotide Transformer were fine-tuned for 50 epochs on variantcentered windows spanning 100 base pairs in either direction to classify case versus control status. The results revealed stark contrasts in performance between the clinical and genomic modeling tracks. In survivability predictions, XGBoost emerged as the superior classifier, achieving an accuracy of 0.74 and an AUC of 0.82, while the XGBoost-Cox model outperformed the traditional Cox model with a C-index of 0.70 compared to 0.66. Through explainability and hazard-based analyses, metastatic site count, tumor mutational burden, the fraction of the genome altered, and the presence of liver and bone metastases were identified as the most potent prognostic indicators across pan-cancer and cancer-specific models. Conversely, the sequence-level transformer models exhibited severe overfitting, with test performance remaining near stochastic levels between 49 percent and 51 percent accuracy. Although DNABERT-2 achieved the highest nominal accuracy at 50.63 percent and HyenaDNA showed superior computational efficiency, the pilot ultimately indicated that fine-tuning transformers on raw sequences in small cohorts is heavily limited by a high signal-to-noise ratio and the polygenic complexity of cancer. Ultimately, this research demonstrates that explainable machine learning models can robustly predict survivability and highlight actionable features for oncology dashboards. However, future sequence-level deep learning efforts must pivot toward using frozen transformer embEd. D.ings or larger, multi-center cohorts to ensure equitable and generalizable clinical adoption.

Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

Electronic health record (EHR) data are inherently multimodal, and leveraging multiple modalities can improve predictive performance. However, most existing approaches rely on deep fusion, which obscures how individual modalities contribute to predictions and limits the interpretability of multimodal reasoning. We propose an explicit multimodal routing framework for clinical prediction that enables interpretable, robust, and auditable reasoning across three EHR modalities: structured longitudinal variables (L), clinical notes (N), and chest X-rays (I). Our model constructs discrete unimodal, directional bimodal, and trimodal routes to capture both individual modality signals and asymmetric cross-modal interactions. To audit multimodal reasoning and assess robustness, we introduce inference-time route masking, which simulates missing modalities and reweights the remaining routes without retraining. We analyze changes in performance and routing weights under these scenarios to understand model decision-making. We evaluate our framework on multi-label phenotype prediction (K = 25) and binary ICU mortality prediction using trimodal patient stays from MIMIC-IV, revealing systematic differences in modality reliance across clinical condition groups. Overall, our framework offers a transparent, auditable, and practical approach to multimodal clinical prediction, providing interpretability, robustness, and insights into how different data sources drive model decisions.