Zheng-Han Ye

Head-Decoupled Logit Distillation Framework for Boosting Logit Transfer Across Tasks

The efficacy of logit transfer in knowledge distillation (KD) is often hampered by disparities in model architectures and imbalances in data distributions. Existing decoupled distillation methods predominantly rely on rigid category partitioning (e.g., target vs. non-target), failing to adapt to varying logit scales ac...