FedProRef: Communication-Efficient Prototype-Count Supervision for Federated Learning Under Severe Label Skew

Abstract

Severe label skew leaves local classifier heads with insufficient supervision for underrepresented classes in federated learning (FL). We present FedProRef, an upload-efficient prototype-count supervision framework for frozen-feature, head-only FL. Each eligible client–class pair uploads only a prototype and sample count. For 512-dimensional features, this replaces a mean-plus-full-covariance payload of 262,656 32-bit floating-point (FP32) values with 513 values, yielding a $512\times $ client-to-server class-statistics upload advantage. The server builds a reusable cross-client anchor pool and learns a class-conditioned residual correction for Gaussian-perturbed proposals; clients use the resulting feature–label pairs to supplement low-count-class head training. Across six CIFAR-100 and Tiny-ImageNet settings with missing local classes, FedProRef averages 75.19% accuracy and improves over Federated Averaging (FedAvg), Federated Feature Augmentation (FedFA), and ProtoAug by 20.39, 14.54, and 2.11 percentage points, respectively. The audited partitions contain 6,682 absent client–class pairs among 9,000 (74.24%), with no empty clients. In a separate coverage-constrained comparison based on the authors’ released code, FedProRef exceeds the Global Geometry-Guided Embedding Uncertainty Representation baseline, denoted GGEUR (aligned), by 13.37 percentage points. Mechanism controls show that clean-anchor supervision provides the principal benefit: DirectAnchorAug and FedProRef both round to 75.19%, while refinement recovers most of the accuracy lost to Gaussian perturbation. With OpenCLIP ResNet-50 (RN50), FedProRef outperforms ProtoAug by 3.03–5.42 percentage points and remains within 0.14 percentage points of DirectAnchorAug. These results support compact prototype-count supervision across transformer- and convolution-based frozen representations.