Teacher-Anchored Worst-case Stability Distillation for Membership Privacy
Knowledge distillation (KD) enables efficient deployment of deep learning models. However, it can increase privacy risks by making the student more vulnerable to membership inference attacks (MIAs), as the student may inherit patterns specific to the teacher’s private training data. Existing KD-based methods against MIAs often rely heavily on reference data or require multiple teachers, which limits their practical applicability. To address this issue, we propose Teacher-Anchored Worstcase Stability Distillation (TASD), a practical single-teacher KD method for reducing membership inference risk. Experiments on CIFAR-10 show that TASD provides stronger resistance to MIAs than existing KD-based methods at lower computational cost.