Hyeon-Jin Jeong

Teacher-Anchored Worst-case Stability Distillation for Membership Privacy

Knowledge distillation (KD) enables efficient deployment of deep learning models. However, it can increase privacy risks by making the student more vulnerable to membership inference attacks (MIAs), as the student may inherit patterns specific to the teacher’s private training data. Existing KD-based methods against MIAs often rely heavily on reference data or require multiple teachers, which limits their practical applicability. To address this issue, we propose Teacher-Anchored Worstcase Stability Distillation (TASD), a practical single-teacher KD method for reducing membership inference risk. Experiments on CIFAR-10 show that TASD provides stronger resistance to MIAs than existing KD-based methods at lower computational cost.

Streamlined Knowledge Distillation

Multi-dimensional Vulnerability Analysis of LM Safety Guards via Role-Goal-Format Decomposition

Current safety guard models for Large Language Models (LLMs) often fail to block complex Jailbreaking attacks because they focus on the structure of a prompt rather than its true intent. To evaluate this problem, we introduce the Role-Goal-Format (RGF) framework, which analyzes how the combination of a user’s persona, their objective, and the text format can bypass safety filters. We compared two small safety guard models: Prompt Guard 2 and Llama Guard 3. The results show that even with the same overall accuracy, they have completely opposite weaknesses. Prompt Guard 2 is easily bypassed by unusual text formats. On the other hand, Llama Guard 3 lacks semantic understanding, missing 76.7% of virus-related attacks. Furthermore, both models exhibit a common vulnerability to contextual breaking; they fail to activate safety alignment when harmful objectives are framed within authoritative and benign personas, such as Teacher, Engineer, and Investigator. These findings highlight that small safety guard models have clear limitations and require multi-layered defenses that understand both text structure and actual intent.