Monte-Carlo methods for some challenging parameter estimation problems: intractable normalizing constants and differential privacy

Abstract

This dissertation studies two statistical problems: Bayesian inference for network models with intractable normalizing constants and differentially private learning with stochastic neural networks.The first project develops noisy Langevin Monte Carlo for exponential random graph models. Auxiliary network simulations estimate the posterior gradient without evaluating the likelihood normalizing constant. The analysis bounds posterior approximation error in second-order Wasserstein distance and the error of estimated posterior expectations. Under the stated curvature, moment, and step-size conditions, short inner Markov chains can suffice as network size increases when parameter dimension grows more slowly than the relevant posterior curvature scale. Simulations and real-network applications assess inferential accuracy and computational performance.The second project develops DP-StoNet, an output-perturbation method for private learning. A constrained latent-variable objective uses quadratic penalties centered at public parameters and feature maps. Parameters and latent variables are optimized without injected training noise; calibrated Gaussian noise is added to the final parameter update before projection. Under the specified assumptions, full-batch training admits Rényi and approximate differential-privacy guarantees, with sensitivity bounded uniformly over the number of iterations. Minibatch training admits approximate differential privacy with either a full-batch or minibatch final update. Nonasymptotic utility bounds account for optimization, sampling, and the single output perturbation. A Music Year experiment studies adaptation from a public model trained on corrupted inputs. In the reported run, DP-StoNet retains the reference model’s predictive accuracy with a small improvement and a lower final test error than the specified DP-SGD baseline under the same target privacy budget.