Aug 2026· Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2· 0 citations· 10 references
TL;DR
Experiments show that LAUA substantially mitigates performance degradation under modality missingness across retrieval and regression tasks, attaining up to 20% relative improvement in MRR for retrieval and up to 24.3% relative improvement in MSE for regression.
Abstract
Multimodal federated learning enables multiple clients to collaboratively train models from distributed multimodal data while preserving data privacy. In realistic federated settings, multimodal samples are often missing or unpaired, and cross-modal heterogeneity across clients can hinder stable optimization. Many existing federated multimodal methods attempt to mitigate modality missingness by generating synthetic paired data through data augmentation or generative models. However, they typically rely on paired supervision or treat client updates uniformly, making them brittle under modality missingness and client-level variability. To address these challenges, we propose a federated multimodal learning framework (LAUA) for learning from a mixture of unimodal and multimodal clients. On clients, LAUA aligns representations in a shared variational latent space, where KL regularization yields a principled and lightweight confidence signal for estimating uncertainty. Unimodal clients learn transferable representations via self-supervised objectives, while multimodal clients additionally leverage task supervision and incorporate an internal distillation component to enhance cross-modal consistency and stabilize local optimization. On the server, LAUA performs uncertainty-weighted aggregation that adaptively down-weights unreliable client updates. Experiments on various datasets show that LAUA substantially mitigates performance degradation under modality missingness across retrieval and regression tasks, attaining up to 20% relative improvement in MRR for retrieval and up to 24.3% relative improvement in MSE for regression.
FedTaste is proposed, a parameter-efficient framework for topology-aware structural transfer in Multimodal Federated Learning with missing modalities that avoids explicit modality imputation while preserving shared semantic structure across clients.
Flux is proposed, a multimodal federated learning framework built around two complementary components, modality-aware confidence tempering and gradient-decoupled private adaptation, that enables sample-specific, client-local confidence adaptation without allowing confidence-dependent gradients to perturb shared representation learning.
Adiba Orzikulova, Jaehyun Kwak, Jaemin Shin et al.· 0 citations
The metrics of individual modality contribution (IMC) and multimodal synergistic gain (MSG) are introduced to quantify sample-level and semantic-level utility, so as to guide semantic denoising selection and robust conditional balancing strategies, effectively mitigating noise interference.
Yan Zhang, Xiaoye Miao, Yanming Yu et al.· Proceedings of the 32nd ACM...· 0 citations
Experiments show that FedAMB improves multimodal accuracy and missing-modality robustness and prevents the propagation of fusion-biased teachers while directly improving unimodal representations.
Seung-Hwa Han, Juyeob Lee, Sang-Min Lee et al.· 0 citations
In this paper, we address the problem of multimodal federated learning with missing modality. Existing methods utilize an additional public dataset or perform naive feature synthesis that is based solely on the available modality. To address these limitations, we propose ProMoE-FL, a Prototype-conditioned Mixture-of-Experts framework for robust missing-modality feature synthesis in multimodal federated learning. ProMoE-FL builds a global client-aware prototype bank that captures clinically meaningful modality priors across institutions. Our Mixture of Experts is conditioned on these prototypes and modality indices to enable direction-aware expert routing for dynamically synthesizing missing features. We perform extensive quantitative and qualitative evaluations on four public chest X-ray datasets (MIMIC-CXR, NIH Open-I, PadChest, and CheXpert) and demonstrate that ProMoE-FL consistently outperforms state-of-the-art methods in both homogeneous as well as the more challenging heterogeneous settings.
Aavash Chhetri, Bibek Niroula, Eduard Vazquez et al.· 0 citations
Federated fine-tuning of Multimodal Large Language Models (MLLMs) across distributed networks enables privacy-sensitive adaptation to evolving data streams, yet a fundamental obstacle prevents robust deployment in dynamic environments: catastrophic forgetting, wherein sequential task updates erase previously acquired knowledge across visual, linguistic, and cross-modal representations. Addressing this challenge is especially critical for autonomous networked AI operating in safety-sensitive domains, such as content moderation, where reliable retention of prior knowledge underpins system integrity. To overcome this, we propose Federated Continual Multimodal Learning (FedCMM), a framework that embeds continual-learning safeguards into the federated optimization loop at three complementary levels. At the parameter level, modality-aware elastic weight consolidation computes separate Fisher information matrices for the vision encoder, language backbone, and cross-modal projector, providing granular, asymmetry-aware protection against modality-specific forgetting. At the data level, each client trains a lightweight local generative replay module to synthesize raw-data-free embedding-level multimodal replay tuples without any raw data sharing. At the aggregation level, Task-similarity-aware gradient aggregation autonomously filters and reweights client updates by gradient cosine similarity, suppressing conflicting directions and stabilizing the global learning trajectory. Extensive experiments on two benchmarks demonstrate that FedCMM consistently outperforms recent baselines on accuracy and backward transfer, confirming that holistic, modality-aware optimization enables robust evolutive adaptation across heterogeneous networked AI deployments.
Jing Liu, Chenxuanyi Zou, Jiayang Ren et al.· 0 citations