Reza Rahimi Azghan

Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning

This work observes that pooled token embeddings from a frozen LLM embedding layer already separate task distributions throughout the learning sequence, and concludes that a Gaussian mixture model fitted on these embeddings, without any gradient-based training, is sufficient for task-agnostic adapter selection at test time, eliminating the need for a learned gating module.