Preserving Fairness in Knowledge Transfer for Exemplar-Free Class-Incremental Learning via Semantic Propagation

Abstract

As a paradigm in continual learning, class incremental learning (CIL) aims to assimilate tasks with mutually exclusive label spaces in sequence while preserving previously established knowledge. Mitigating forgetting in CIL fundamentally relies on transferring knowledge across tasks. A straightforward exemplar-based approach promotes balanced knowledge transfer by replaying an equal number of samples from each old class. However, in the more challenging exemplar-free setting, this balance cannot be ensured because distillation-based cross-task knowledge transfer tends to focus more heavily on the knowledge acquired from the most recent tasks. To address the unfairness in knowledge transfer, we analyze the mechanisms underlying dark knowledge and introduce a Semantic Enhanced Knowledge Transfer (SEKT) method for exemplar-free CIL. Specifically, SEKT adopts a bi-flow framework. The first flow is the Semantic Guidance Flow (SGF), which is inspired by knowledge distillation and produces latent semantic distributions from the outputs of the old model to guide the new model toward generating similar distributions. The second flow is the Semantic Propagation Flow (SPF), which propagates latent early knowledge to the current task in order to mitigate the unfairness in knowledge transfer. SPF constructs a cross-task semantic similarity graph using aligned intermediate representations to enable semantic propagation. It employs an expert network to learn the pattern of semantic propagation, enabling real-time and stable semantic recovery during training. In contrast to the SGF that is more effective for transferring recent knowledge, the SPF learns complementary early knowledge through a semantic complementarity constraint. Moreover, the SPF is robust to noisy semantics, as the learned semantic distribution is regularized with an $\ell _{2,1}$ norm. Extensive experiments conducted on six datasets demonstrate the superiority of the proposed SEKT over existing exemplar-free CIL approaches.