Demystifying and unifying ELBO for Probabilistic Deep Learning Inversion

Abstract

Variational inference has emerged as a computationally efficient framework for solving Bayesian inverse problems. Rather than relying on random sampling in high-dimensional model spaces, it reformulates probabilistic inference as an optimization problem, marking a fundamental shift from sampling-based to optimization-based approaches. Its objective, the evidence lower bound (ELBO), is central to modern deep generative approaches, but its form can look different from one model class to another. This article derives the ELBOs step by step for two representative models used in probabilistic geophysical inversion: conditional variational autoencoders (cVAEs) and normalizing flows (NFs). The cVAE objective appears as a reconstruction term minus a latent-space Kullback–Leibler (KL) divergence, whereas the NF objective appears as data likelihood plus prior plus entropy. We show that these are not separate principles. Both are direct substitutions into the same generic ELBO identity. The apparent difference comes from the modeling choice of what counts as the observed variable and what is the hidden variable: in cVAE training, the hidden variable is the latent code; in unsupervised NF inversion, the hidden variable is the Earth model itself.