Vectorless Retrieval-Augmented Generation: A BM25-Based Alternative to Embedding-Driven RAG Systems

Abstract

Retrieval Augmented Generation (RAG) augments large language models using external knowledge at the inference stage to increase factual accuracy and reduce hallucinations. Most existing RAG models employ dense vectors for information retrieval, which facilitates semantic matching at the expense of increased computation costs, latency, and lack of transparency. In our paper, we propose a novel vectorless RAG system which replaces embedding-based retrieval with conventional information retrieval methods such as full-text matching, BM25 ranking, and keyword matching. We conduct experiments comparing vector-based and vectorless models under identical conditions in terms of the underlying document pool, pre-processing steps, and language model. The comparison is carried out using several performance criteria such as latency, faithfulness, diversity, keyword alignment, and semantic relevancy. Experiment results show that the proposed vectorless model achieves competitive performance while providing advantages in terms of efficiency, interpretability, and grounding. However, it exhibits limitations in handling semantically complex queries. These findings suggest that vectorless retrieval is well-suited for precision-driven and resource-constrained applications, and that hybrid approaches may further enhance performance.