Coordination on a Budget: Federated Active Learning with Few Labels

Adaptive Ranks for Personalized Federated Large Language Models Under Parameter Budget Constraints

The advent of Large Language Models (LLMs) represents a leap toward leveraging AI for the benefit of humanity. However, the realization of this potential requires addressing the growing need for personalization and data privacy across diverse entities. Federated Learning (FL) offers a vital paradigm for developing private and personalized LLMs, yet its effectiveness is often limited by the inequality of computational resources at the edge. Existing federated Low-Rank Adaptation (LoRA) struggles to accommodate such heterogeneous hardware capacities, leading to information loss during model aggregation. To bridge this gap, this paper presents an adaptive aggregation framework designed to optimize federated LLM fine-tuning under these constrained conditions. The framework introduces a dynamic budgeting mechanism that quantifies personalization intensity through Frobenius norm divergence and allocates rank capacity accordingly. To resolve rank heterogeneity, a dimension-aligned strategy based on Singular Value Decomposition (SVD) is applied, enabling the consistent fusion of updates across diverse devices. Extensive experiments on the General Language Understanding Evaluation (GLUE) benchmark using Llama-3-8B achieve an average score of 0.6551 and a mean task ranking of 2.22. The results indicate that our method improves global generalization while preserving client-specific personalization in resource-limited environments.

Collaborative Synthetic Data Generation for Knowledge Transfer in Federated Learning

One-shot federated learning (OSFL) addresses the communication overhead of federated learning by limiting training to a single round, but doing so without sacrificing model quality is non-trivial, particularly when client data distributions diverge. Recent work has addressed this challenge by aggregating client knowledge on the server through the construction of transferable synthetic datasets or distillates. However, most of these methods lack formal privacy guarantees, leaving a gap in jointly achieving low communication, robustness to heterogeneity, and rigorous privacy. We propose FedKT-CSD (Federated Knowledge Transfer via Collaborative Synthetic Data), a framework inspired by neural image compression that closes this gap by leveraging publicly pretrained autoencoders as a shared latent space. Each client encodes its private data in a single forward pass, computes class-conditional latent statistics, and transmits these to the server. The server aggregates these statistics via secure aggregation, adds calibrated differential privacy noise, and decodes a synthetic dataset for training a global model and further downstream tasks. This design provides formal $(\varepsilon,\delta)$-differential privacy by construction, while keeping client-side computation and communication lightweight. Despite operating under privacy constraints, FedKT-CSD is competitive with and even outperforms non-private baselines across diverse datasets and heterogeneity settings, and scales to a large number of clients. Our code is available at: https://github.com/an7123/FedKT-CSD

Aggregate-Only Observability for Privacy–Utility Tradeoffs in Federated Learning

Federated Learning (FL) avoids centralizing raw data, but server-side access to per-client updates still creates a significant privacy risk because gradients can leak sensitive information through inversion and related attacks. A common defense is client-level Differential Privacy (DP), which reduces attack fidelity by perturbing individual updates, but often at the cost of degraded predictive utility, especially in heterogeneous settings. In this paper, we study a different design choice: aggregate-only observability, where the server learns only an aggregate update rather than the individual client contributions that compose it. We develop a unified evaluation framework comparing baseline FL, client-level DP-FL, an idealized sum-only aggregation mechanism (hide_sum), and a quantum-inspired aggregate-observation primitive (qsum) based on finite-shot GHZ-style phase estimation. Experiments on Texas PUDF and MIMIC-IV show that the aggregate-only regimes preserve competitive predictive utility relative to baseline FL while removing the direct per-client observation channel used by standard server-side inversion attacks. We also analyze utility sensitivity to shot budgets and the resource scaling of the quantum-inspired primitive. The results suggest that privacy–utility tradeoffs in FL depend not only on how updates are perturbed, but also on what the server is allowed to observe. Aggregate-only observability therefore provides a useful mechanism-level complement to noise-based privacy defenses, while also motivating hybrid designs to address residual leakage from aggregate updates.

FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgetting

FedADB, a Class Anchor-Driven Dual-Branch FL framework, a dual-branch collaborative training mechanism designed for clients that achieves significant improvements in both accuracy and convergence speed.

Beyond Client Clustering: Fine-Grained Preference Alignment in Federated RLHF via Self-Evolving Routing

Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learning

Sub-model federated learning lets resource-constrained clients train width-reduced versions of a global model, but existing methods allocate capacity by device resources alone. A natural next step, allocating capacity by each client's data heterogeneity as estimated from the updates the server already observes, has been repeatedly suggested. We ask whether that step is possible, using HAS-FL, an adaptive capacity-allocation framework, as a test case. Our findings are threefold. First, validated against ground-truth label-distribution divergence on reproducible partitions, update-divergence estimates of client heterogeneity are dominated by capacity rather than data: across two corrected estimators, multiple datasets, and all seeds, the estimates correlate strongly and negatively with device capacity, and no data signal remains once capacity is controlled for. This previously undocumented confound affects any method estimating client statistics from sub-model updates. Second, adaptive allocation has a hidden failure mode: when every client is capped below full width, the uncovered parameters stay at random initialization and progressively corrupt the global model. A simple coverage guarantee removes the failure and explains why uniform allocation collapses. Third, a matched-budget control settles what adaptivity contributes: random allocation to the same average budget performs no differently on both image benchmarks, and on the naturally partitioned text benchmark the adaptive policy is the weakest of the three strategies while consuming the most capacity. Sub-model training remains valuable because it admits constrained clients at quadratically reduced cost, but what protects accuracy is parameter coverage rather than allocation intelligence. Its apparent benefits come from capacity budgeting and coverage, and future designs need heterogeneity signals separable from capacity effects.