Research on Attention-Based Dynamic Differential Privacy Machine Learning Methods

Abstract

Amid the rapid proliferation of big data and machine learning technologies, concerns around data privacy breaches have grown increasingly acute. Differential privacy offers a mathematically rigorous framework for privacy preservation and has found broad adoption in a range of machine learning settings. Still, conventional approaches tend to rely on static, uniform noise injection strategies—failing to account for the fact that different features and stages of training may have vastly dissimilar privacy sensitivities. As a consequence, privacy budgets are often used inefficiently, and model utility suffers noticeably. In response to these limitations, we introduce an attention-driven dynamic differential privacy mechanism that enables adaptive allocation of privacy budgets. Our design aims to uphold strong protection without sacrificing model performance as much as before. Specifically, we build a lightweight multi-head attention component that dynamically evaluates each feature’s relevance and the current phase of training, adjusting the intensity of injected noise on the fly. This component is designed for easy integration into existing deep learning pipelines without major structural modifications. We evaluate our method on the MNIST benchmark, and results show clear improvements in classification accuracy under identical privacy constraints. Further, we provide visual evidence through attention heatmaps and noise variation trajectories, which together offer interpretable support for the adaptive mechanism's effectiveness.