Preserving Membership Privacy via Risk Score Guided Knowledge Distillation

Abstract

: The increasing deployment of deep learning models has raised important concerns regarding data privacy. In particular, Membership Inference Attacks (MIAs) aim to determine whether a specific data sample was used to train a model, potentially exposing sensitive information. To address this issue, we propose Risk Score guided Knowledge Distillation (RS-KD), a defense mechanism that dynamically adapts the distillation process according to a privacy risk score estimated from the output characteristics of a teacher model. This risk estimation enables the identification of samples that are more vulnerable to MIAs, allowing the framework to selectively regulate the uncertainty introduced during distillation. Experimental results on benchmark datasets demonstrate that RS-KD significantly reduces the effectiveness of black-box MIAs while preserving predictive performance, achieving a favorable privacy–utility trade-off.