Skip to content

Prior-Amortized In-Context Bayesian Inference for Generalized Linear Mixed-Effects Models

Sep 2026 · 0 citations
Computer Science Mathematics

Abstract

Hierarchical data is ubiquitous in the empirical sciences and is most commonly analyzed with generalized linear mixed-effects models (GLMMs). Bayesian inference for GLMMs yields calibrated uncertainty but requires MCMC; the No-U-Turn Sampler (NUTS) is the gold standard but is slow and must restart from scratch for every new dataset, model and prior. We introduce metabeta, a pretrained neural network for prior-amortized in-context Bayesian inference over GLMMs. Unlike previous neural posterior estimators that fix the prior at training time, metabeta accepts prior families and hyperparameters as inputs at test time, enabling zero-shot generalization. Two set transformers and conditional normalizing flows mirror the posterior's two-level structure (global parameters shared across groups, local parameters per group). The model is trained on millions of realistic simulated datasets spanning continuous, binary, and count outcomes. By default, the flow posterior is refined by Independence Metropolis-Hastings against the unnormalized posterior, so its correctness rests on the sampler rather than the network; this yields tuning-free inference two to three orders of magnitude faster than NUTS. Alternatively, the flow can warm-start NUTS, giving nearly identical inference with substantially increased speed and stability. On controlled benchmarks with ground-truth parameters, metabeta matches NUTS in parameter recovery, calibration and out-of-sample prediction. On out-of-distribution real datasets, its posteriors closely match those of NUTS across all parameter types, and they remain faithful under misspecified likelihoods and priors, out-of-distribution predictors, collinear designs, and data-poor regimes. The model is open-source and open-weights and thus immediately deployable.

View source

Similar papers

#machine learning Preprint Oct 2026

LinearPFN: Amortized Variable Selection for Linear Models with Interactions

Spike-and-slab regression is a standard Bayesian formulation of variable selection: it returns a posterior distribution over which candidate effects are active rather than a single selected subset, so that every candidate effect carries an inclusion probability. Its cost grows exponentially with the number of candidate...

Louis Schiekiera, Max Zimmer, Christophe Roux et al. · 0 citations

Aggregated Posterior Predictive Checks for Generative Modeling

Experiments with variational autoencoders show that aggregated posterior sampling improves generation for heavy-tailed and clustered data relative to Gaussian prior sampling while performing comparably to models with more flexible latent priors.

S. Dutta, Gemma E. Moran · 0 citations
#machine learning Preprint Sep 2026

Amortized Bayesian Inference on Multilevel Models of Arbitrary Structure

We develop a general method for amortized Bayesian inference on multilevel models of arbitrary structure. Given a generative model specified as a directed acyclic graph, our method automatically derives valid factorizations of the joint posterior and matching neural network architectures. The key steps, graph expansion...

Daniel Habermann, Andreas Bulling, Stefan T. Radev et al. · 0 citations
Preprint Sep 2026

Subsampled Pseudo-posteriors for Scalable Bayesian Moment-condition Inference

A new family of subsampled pseudo-posteriors is developed that replaces the full-data pseudo-posterior with an aggregate of mini-batch pseudo-posteriors and establishes finite-sample total variation bounds to the full-data pseudo-posterior, allowing discontinuous and intractable moment functions.

Wen-Shuo Zhao, Rong Tang · 0 citations
Preprint Oct 2026

Covariate-dependent Nonparametric $g$-modeling for regression via infinite Mixture-of-Expertizing class

Empirical Bayes $g$-modeling captures unit-level heterogeneity by estimating a latent prior distribution from observed data. In the existing formulations, however, the prior is shared by all units. In this paper, we develop a covariate-dependent g-modeling framework for regression in which the entire prior distribution...

Akira Okazaki, Keisuke Yano · 0 citations
Preprint Sep 2026

Bayesian Posterior Learning of Mixed Graphical Models

The Mixed Graph WWA method is proposed, a class of MCMC methods for posterior simulation in Bayesian MGMs based on the Birth-Death MCMC methodology that achieves graph recovery accuracy comparable to or better than existing approaches, while maintaining efficient posterior exploration and favorable effective sample siz...

Er-Dong Guo, A. Beskos, M. De Iorio · 0 citations

Related blog posts

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.