Shu-Hao Guan

Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models

This work reformulates the deterministic LoRA update as a probabilistic low-rank representation inspired by Sparse Gaussian Processes, and identifies a structural isomorphism between LoRA's factorization and Kronecker-factored SGP posteriors, and shows that LoRA emerges as a limiting case when posterior uncertainty col...