Preprint
Jul 2026
Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning
This work observes that pooled token embeddings from a frozen LLM embedding layer already separate task distributions throughout the learning sequence, and concludes that a Gaussian mixture model fitted on these embeddings, without any gradient-based training, is sufficient for task-agnostic adapter selection at test time, eliminating the need for a learned gating module.
Reza Rahimi Azghan, Gautham Krishna Gudur, Giulia Pedrielli et al.
· 0 citations