Skip to content
Book Open access

Pre-retrieval Query Clustering for Adaptive Top-k Document Retrieval in RAG Systems

Sep 2026 · Proceedings of the 35th ACM International Conference on Information and Knowledge Management · 0 citations · 28 references
Computer Science

TL;DR

This approach is pre-retrieval and query-centric, making it robust in heterogeneous, case-like corpora and applicable across domains such as legal, healthcare, finance, and enterprise search.

Abstract

RAG systems commonly retrieve a fixed number of documents (top-k) to ground generation, but this static approach is brittle: simple queries suffer over-retrieval (adding noise and cost) while complex queries are under-retrieved, causing recall failures that cascade into incorrect answers. Motivated by the question of how many documents must be retrieved to answer an arbitrary query reliably, we propose a practical, general framework for query-adaptive retrieval depth. Offline, we estimate per-query retrieval difficulty by measuring NDCG under the default retriever and deriving a query-specific ''saturation'' point k* from the NDCG–k curve. Because computing these signals online is expensive, we cluster a large set of queries in embedding space and summarize each cluster with a recommended retrieval depth that targets high coverage (e.g., ~95%) using a mean-plus-variance rule. At runtime, the system assigns an incoming query to a cluster and selects the corresponding top-k in constant time. Compared with post-retrieval confidence methods that rely on clustering retrieved documents, our approach is pre-retrieval and query-centric, making it robust in heterogeneous, case-like corpora and applicable across domains such as legal, healthcare, finance, and enterprise search. Finally, this framework has been tested in full-traffic queries that improved F1 by over 36% while reducing token usage by 14% on low-complexity clusters without accuracy loss.

Read PDF

Similar papers

Preprint Sep 2026

Top-K Is Not a Budget for Hybrid Retrieval

DiBud is presented, which takes an access budget directly as input and incrementally certifies and returns an exact prefix of the RRF ranking over the full lists, and investigates long-tailed costs for completing exact Top-20.

Chunran Zhang · 0 citations
#artificial intelligence Preprint Sep 2026

ViSAR: Training-Free Adaptive-$k$ Retrieval for Visual Document Question Answering

ViSAR (Visual Semantic Activation Retrieval), a training-free adaptive-$k$ retrieval method for late-interaction visual document retrieval, is introduced and it is shown that the similarity matrix structure correlates with answer accuracy, suggesting future directions for retrieval quality-aware document understanding.

Adrien Mialland, Marc Plantevit, Julien Gallois et al. · 2 citations
#artificial intelligence Open access Sep 2026

ORDER: Task-Conditioned Routing for Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) pipelines typically rely on a fixed indexing and retrieval configuration determined at preprocessing time. This one-size-fits-all design is ill-suited to domain-expert settings, where heterogeneous queries require different chunking granularities, metadata constraints, and source-se...

Aurélien Pellet, Marie Puren, Julien Perez · 0 citations

A neural network approach to adaptive document retrieval in rag systems

Dynamic-K (DYK), an adaptive retrieval depth mechanism that determines the number of documents for each query, is proposed, which improves retrieval of relevant documents and reduces missed important evidence, particularly for highly relevant documents.

Neennara Rodnit · 0 citations

A Bag-of-Documents Model for Query Specificity ⋆

Bag-of-Documents Specificity (BoDS) is introduced, a two-stage approach that operationalizes query specificity as intent coherence : how tightly a query’s engaged documents cluster in embedding space using the mean resultant length of a query’s associated document vectors.

Aritra Mandal, Daniel Tunkelang, Zhe Wu · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.