Revisiting Assumptions for Membership Inference on Summary Statistics
Research studies routinely publish summary statistics such as means and standard deviations to promote transparency while protecting participant privacy. Membership inference attacks (MIAs) can exploit these statistics to determine whether a specific individual contributed to a study, posing a risk especially in biomedical and health-related settings. However, existing attacks assume the adversary holds the exact data used in the study, an assumption that rarely holds when data evolves over time. Moreover, prior work has not quantified how much of the reported accuracy stems from true individual identification rather than from group-level traits shared within disease cohorts. We investigate the robustness and interpretability of two standard attacks—the L1-distance test and the log-likelihood ratio (LLR) test—under realistic conditions where the adversary has only noisy, partial, or temporally mismatched data. We derive a theoretical lower bound on inference error that cleanly separates a statistical term governed by pool size and feature dimensionality from a signal term capturing disease-driven shifts. Empirical evaluation on cross-sectional and longitudinal miRNA datasets, validated on Fitbit activity data, confirms that both attacks tolerate substantial noise and missing features, but that real-world temporal drift degrades accuracy far more steeply than synthetic perturbations predict, and that this degradation is individual-specific. We further show that attack accuracy on disease-specific cohorts exceeds that on size-matched random pools by approximately 10%, a separation that grows almost threefold when measured by true-positive rate at 1% false-positive rate. Moreover, individuals sharing disease traits but absent from the study are frequently misclassified as members, indicating that a substantial component of reported accuracy reflects shared condition rather than individual membership.