RightFeatKD: Selective Feature-based Knowledge Distillation
RightFeatKD is proposed, a novel feature-based distillation framework that captures rich semantic knowledge from the intermediate representations of the teacher model using a specialized loss function, ensuring reliable and meaningful supervision.
Syed Tousiful Haque, Yan Yan, Anne Hee et al.
· 0 citations