Skip to content
Log in
Preprint

ADS-C: Antidistillation Sampling for Classification

Khawaja Abaid Ullah M. J. Khojasteh
Jul 2026 · 0 citations · 71 references
Computer Science

Abstract

Knowledge distillation enables an adversary to replicate a proprietary classifier by querying its prediction interface and training a surrogate on the returned probability vectors. Antidistillation sampling, proposed for large language models, counters this threat with an input-dependent, gradient-directed perturbation of the served distribution; its transfer to classification has not been studied. Adapting the defense to classification, we show its behavior is governed by the distribution of the teacher's per-input confidence margins. Because well-trained classifiers are severely overconfident, the direct transfer exhibits an inert window: below a closed-form-predictable threshold, it affects neither attacker nor defender; beyond it, the defense undergoes a phase transition and degrades the teacher faster than the attacker's student. Temperature softening rescales the transition in closed form, and every temperature configuration lies on the same unfavorable trade-off curve. Our method, ADS-C, composes the perturbation under a closed-form, per-input margin budget that provably preserves every served top-1 prediction, so the defended teacher's accuracy equals the undefended teacher's identically. Under this guarantee the distilled student still loses 17.4 percentage points on CIFAR-100, 29.6 on CIFAR-10, and 13.3 on Tiny-ImageNet; matching this degradation with the unmodified defense costs 27.5, 32.9, and 22.2 points of teacher accuracy. Because served labels are unchanged, a hard-label attacker gains nothing, while the defended soft output trains a student up to 29.7 points below that floor: the incentive to distill served probabilities is not merely removed but reversed. To our knowledge, ADS-C is the first antidistillation defense for classification whose utility cost is exactly zero.

arXiv Semantic Scholar
View source

Similar papers

Preprint Aug 2026

Caliber: Cross-Architecture Extraction-Cost Control for Score-Returning APIs

We present Caliber, an output-perturbation defense against model extraction that formulates noise selection as a calibration problem: how much the defense degrades the supervision signal used to train a surrogate, and the provable per-input query cost of recovering the clean logits. To defend against an attacker that uses returned scores for knowledge distillation, Caliber adds independent and identically distributed Gaussian noise to the internal logits. We establish two properties of the resulting perturbed predictions. Monotone agreement degradation: When the clean logits have a unique maximizer, agreement with the clean prediction decreases strictly with the noise scale, so every target in $(1/K,1)$ corresponds to a unique positive scale; task accuracy is bounded by computable lower and upper envelopes. Per-input recovery cost: We derive a closed-form minimax lower bound on the repeated queries needed to recover the clean logits for a fixed input. Caliber normalizes noise variance by the squared median top-two logit margin and fits the resulting noise-utility relationship with a logistic curve, either per model or shared within a task. Across more than thirty model-dataset combinations, per-model calibration achieves mean absolute relative errors of 0.6-1.4%. End-to-end experiments show that surrogate performance generally tracks the configured degradation, while fixed-input averaging follows the expected variance reduction.

Chi Wang, Hanwen Wang, Yuchen Xia et al. · 0 citations
Preprint Jul 2026

Anti-Backdoor Coreset Selection via Cumulative Entropy

This paper formulate this defense strategy as a coreset selection problem, giving rise to so-called anti-Backdoor Coreset Selection, and uses the Cumulative Entropy as selection criterion to further facilitate this effect.

Qi Zhao, Christian Wressnegger · 0 citations
Jun 2026

The Platonic Defense: Backdoor Defense for Self-Supervised Encoders in the Era of Large Scale Pre-training

A new attack-agnostic, model-agnostic, and modality-agnostic black-box test-time defense paradigm, inspired by the Platonic Representation Hypothesis, that can perform both representation detection and purification, and achieves substantial performance gains across multiple attacks.

Tuo Chen, Minjing Dong, Benlei Cui et al. · 0 citations
Conference Open access 2026

Stop Hardening Everything: A Training-Free Neuron-Level Defense for Neural Ranking Models

This paper introduces Search in the Model, a novel training-free framework that performs fine-grained identification and rectification of vulnerable neurons directly within the model, and develops a maximum marginal vulnerability criterion to precisely locate the top-K neurons most responsible for model vulnerability.

Yuansan Liu, Ruqing Zhang, Hongru Song et al. · 0 citations