Dynamic-K (DYK), an adaptive retrieval depth mechanism that determines the number of documents for each query, is proposed, which improves retrieval of relevant documents and reduces missed important evidence, particularly for highly relevant documents.
Abstract
This study investigates retrieval depth determination in Retrieval-Augmented Generation (RAG) systems for medical applications. While RAG improves answer generation by incorporating retrieved documents, most systems use a fixed number of documents (Fixed-K), regardless of query complexity. This limitation can lead to false negatives, where relevant documents are not retrieved, resulting in incomplete answers—especially critical in medical contexts. To address this issue, this study proposes Dynamic-K (DYK), an adaptive retrieval depth mechanism that determines the number of documents for each query. The method leverages structural characteristics of ranked document lists produced by a hybrid retrieval pipeline, and employs a Multi-Layer Perceptron (MLP) to learn the appropriate retrieval depth. The pipeline integrates BM25, dense retrieval, and cross-encoder reranking. Experiments on the NF Corpus evaluated using Recall@K, Precision@K, and F1@K show that Dynamic-K improves retrieval of relevant documents and reduces missed important evidence, particularly for highly relevant documents. These findings suggest that adaptive retrieval depth is an important additional dimension in RAG system design, with strong potential for medical question answering applications.
This approach is pre-retrieval and query-centric, making it robust in heterogeneous, case-like corpora and applicable across domains such as legal, healthcare, finance, and enterprise search.
Ye Xia, Emre Yamangil, Hai-Xun Wang· Proceedings of the 35th ACM...· 0 citations
ViSAR (Visual Semantic Activation Retrieval), a training-free adaptive-$k$ retrieval method for late-interaction visual document retrieval, is introduced and it is shown that the similarity matrix structure correlates with answer accuracy, suggesting future directions for retrieval quality-aware document understanding.
Adrien Mialland, Marc Plantevit, Julien Gallois et al.· 2 citations
It is concluded that diversification should be used selectively, based on observable redundancy and evidence requirements, and introduces a query-adaptive rule that diversifies only when the effective number of distinct documents in the nearest-neighbor top-$k$ selection falls below the query's evidence requirement.
Guillaume Brouillette, Faustin Kagabo, Usef Faghihi et al.· 0 citations
Correct page selection, rather than semantic retrieval in isolation, is identified as the main opportunity for improvement in this setting of retrieval-augmented generation, with higher latency and more detected output artifacts.
An Intelligent Document Processing Platform using Retrieval-Augmented Generation to enable accurate, context-aware, and reliable document intelligence and offers a practical and scalable framework for intelligent document understanding, semantic search, and AI-assisted question answering in modern knowledge management...
J. Priya, M. Arathi· International Journal for Re...· 0 citations
The main implication of this research is the validation of a practical architecture for improving IR systems, offering a viable alternative for domain-specific contexts such as Sequran.
Ray Ramadita, Wisnu Uriawan, W. Zulfikar· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.