A Meta-Learning-Based Aggregation Strategy for Heterogeneous Federated Learning Scenarios
TL;DR
It is demonstrated that prediction-space meta-learning constitutes a practical alternative for federated aggregation without requiring shared parameter initialization, and among the different forms of heterogeneity in federated learning, this work focuses on statistical and model heterogeneity.
Abstract
Federated learning relies on aggregation schemes that assume all participants train models with identical architectures and a common parameter initialization. While this enables parameter-averaging strategies such as Federated Averaging, it also imposes a strong inductive bias by constraining local models to evolve from nearly identical starting points, potentially reducing model diversity and limiting exploration of the hypothesis space. Recent studies have explored shared-initialization-free and heterogeneous federated learning as largely independent research directions. In this context, we propose a shared-initialization-free, architecture-agnostic aggregation strategy based on meta-learning, where local models generate predictions over a reference dataset that are integrated to train a global meta-model. Among the different forms of heterogeneity in federated learning, this work focuses on statistical and model heterogeneity. The proposed framework is evaluated in both one-shot and multi-round federated settings against parameter-averaging and knowledge-distillation approaches while preserving the independent evolution of local models. Experiments across diverse federated settings, including homogeneous and heterogeneous models, varying numbers of participants, different levels of data imbalance on representative benchmark dataset demonstrate that the proposed strategy in a one-shot scenario achieves competitive performance under limited-data conditions, while iterative refinement of the prediction consensus in the multi-round setting yields consistent improvements over traditional aggregation strategies. These findings demonstrate that prediction-space meta-learning constitutes a practical alternative for federated aggregation without requiring shared parameter initialization. Future work will investigate more advanced consensus mechanisms, adaptive historical consensus strategies, and the evaluation of the proposed framework under larger-scale federated learning environments.