Efficient Continual Adaptation of Generative Retrieval Systems over Dynamic Corpora

Abstract

Generative Retrieval (GR) encompasses two primary paradigms: Generative Document Retrieval (GDR), which directly generates document identifiers, and Retrieval-Augmented Generation (RAG), which synthesizes reliable responses from retrieved evidence. Despite their promising performance, most existing GR approaches assume static corpora. In GDR, corpus knowledge is encoded directly into model parameters, meaning that integrating new documents requires expensive retraining while risking catastrophic forgetting. Non-stationarity also arises along the temporal dimension: in RAG systems, facts evolve and queries frequently contain temporal constraints. When temporally misaligned evidence is retrieved, generation quality degrades regardless of the LLM's reasoning capability, demonstrating that the retriever forms a central bottleneck under evolving knowledge. Beyond textual corpora, modern Information Retrieval (IR) increasingly operates over multimodal collections that combine text, images, and other modalities. These corpora are likewise dynamic and subject to continuous distributional shifts. In summary, real-world information retrieval systems are characterized by multiple forms of non-stationarity: text corpus growth, temporal evolution, and multimodal corpus growth. This thesis investigates principled and efficient continual learning (CL) mechanisms for GR under these realistic conditions, with a unifying hypothesis that scalable and efficient retriever adaptation is essential to maintaining effective GR.