This work presents a hierarchical multimodal MoE for interstitial lung disease (ILD) classification that integrates a frozen, pre-trained imaging expert with structured electronic health records (EHR) via two-stage gating via two-stage gating.
Abstract
Mixture-of-experts (MoE) models combine specialized predictors under learned routing, offering a principled mechanism for leveraging heterogeneity in medical data. We present a hierarchical multimodal MoE for interstitial lung disease (ILD) classification that integrates a frozen, pre-trained imaging expert with structured electronic health records (EHR) via two-stage gating. A modality-level gate assigns patient-specific weights to imaging and EHR predictions, while a sub-gating module decomposes the EHR branch into clinically defined feature groups with learned, group-specific contributions. This design preserves stable imaging representations while enabling input-dependent clinical weighting and explicit EHR specialization. Under strict patient-level cross-validation, the model achieved the highest mean AUC among the evaluated methods (0.8750 +- 0.0443), compared with 0.8646 for imaging-only REN and 0.7685 for SwinUNETR. The framework extends interpretability across anatomical regions, imaging--EHR utilization, and clinically defined EHR feature groups.
A novel three-axis taxonomy covering Gating Architecture, Application Level, and Supporting Techniques is proposed and a controlled comparison of six gating architectures on three biomedical benchmarks shows that the five Axis 1 gating types produce measurably distinct performance profiles and that no single architectu...
Ba-Duy Nguyen, Van-Dung Hoang, Hien D. Nguyen· Network Modeling Analysis in...· 0 citations
Combining whole-body magnetic resonance imaging (WB-MRI) with clinical variables has the potential to improve systemic disease diagnosis by leveraging complementary sources of patient information. However, structured clinical variables are often incomplete or missing, limiting the applicability of conventional multimod...
L. Daza, M. Hasny, Cristina González et al.· 0 citations
Multimodal medical data provide complementary information for cancer diagnosis and prognosis; however, effectively integrating heterogeneous modalities remains challenging because real-world clinical datasets frequently contain incomplete modality observations. We present a unified benchmarking framework for multimodal...
Nhu Le Nguyen Quynh, Phuong Nguyen Ngoc Thanh, Tuan Le et al.· International Conference on...· 0 citations
Selective Cross-Modal Learning is introduced, a lightweight pre-fusion refinement framework that enriches unimodal embeddings by selectively incorporating complementary cues from concurrently available modalities and maintains its advantage under missing-pathology and missing-genomics conditions.
Mohammed Belghadi, Younès Bennani, Valérie Paradis· 0 citations
Recent advances and emerging translational evidence for clinically oriented MML in glioblastoma is summarized, with particular emphasis on MRI-centered systems that integrate imaging with pathology, selected molecular measurements, and longitudinal clinical context.
Amin Zadeh-Shirazi, Bryan W. Day, Hui K. Gan et al.· Frontiers in Oncology· 0 citations
This work proposes MultiSigBERT, a unified framework for multimodal sequential survival modeling in oncology based on path signature representations that achieves a concordance index of 0.743 on an independent test set, demonstrating the benefit of jointly modeling multimodal temporal dynamics together with patient-lev...
Paul Minchella, Stéphane Chrétien, Guillaume Metzler et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
