On-Device Is Not Private: Measuring Membership Leakage in Lightweight Sensor Anomaly Detectors

Abstract

Compressed-TinyML pipelines apply distillation, quantization and pruning to fit lightweight anomaly detectors onto resource-constrained sensor nodes. Such frameworks are routinely called privacy-aware on the strength of on-device inference or noisy federated updates rather than any measured guarantee, yet that assumption is rarely tested with a membership-inference attack. Across six multivariate time-series datasets (sensor, facility and finance) and four compact reconstruction-autoencoder architectures (84 experimental cells), a detector accurate enough to deploy (detection AUC 0.878) is vulnerable to a reconstruction-error membership-inference attack (attack AUC 0.688; true-positive rate 0.074 at a 1% false-positive rate). A gradient-noise defense (gradient clipping with additive Gaussian noise) reduces measured leakage to near chance (attack AUC 0.508 as scored, 0.517 oriented away from chance, not exact chance; TPR cut 4.8-fold) at a utility cost of 0.054 in detection AUC (paired Cohen’s d=4.06 for the leakage drop, bootstrap 95% CI [0.145,0.210]), consistently across all datasets (one-sided paired Wilcoxon p=0.0156), while the utility change is nominal (p=0.031, d=1.36). Exported and executed as a real INT8 ONNX artifact, the model shrinks 3.2× but shows no evidence of improved privacy (mean leakage rises slightly; improvement test p=0.95, worsening test p=0.078, neither significant) and runs slower on average on a desktop CPU, so quantization is a storage optimization only. The primary attack is the loss-threshold attack; a shadow-model check on a subset reaches the same qualitative conclusion. The defense is not formal differential privacy, but the finding is clear: on-device processing does not imply privacy; the privacy of a compressed sensor detector must be measured, not assumed.