A Federated Learning and Knowledge Distillation-Based Load Identification Algorithm Under Heterogeneous Model Condition

Abstract

With the development of power system intelligence and information, load identification technology has played an important role in smart electricity utilization. Although traditional centralized load identification methods can improve model accuracy, they inevitably lead to increased risks of privacy leakage and high communication costs. In order to cope with the challenges of data privacy protection, model heterogeneity, and communication efficiency in the field of load identification, a federated learning (FL)- and knowledge distillation-based load classification algorithm under heterogeneous models is proposed in this paper. First, the measurement is transformed into an RGB image using Markov random fields, which not only provides more information but also satisfies the requirement of deep networks, so the actual load identification problem is formulated as an image classification problem. Then, the proposed FL framework is described. Each client uses different image classification networks for identification with its own computing power and requirements. In local training, the fine-tuning strategy is used where the parameters of feature extraction are frozen and only the classification parameters are trained. By introducing public data as a distillation carrier, each client can generate soft labels of the public dataset to represent the knowledge of their own data. At the server, knowledge sharing between datasets is achieved through aggregation. Both the soft labels and the trained parameters are aggregated by their corresponding the training performance. The proposed algorithm only transmits soft labels and classification parameters, which can ensure user privacy and reduce communication traffic. The experimental results demonstrate that the proposed algorithm improves the load identification performance under the WHITED and PLAID dataset conditions, compared with some existing methods.