Sep 2026· International Journal of Innovative Science & Technology· 0 citations· 13 references
TL;DR
This study tested how both settings change the results of a simple RAG pipeline, finding retrieval recall was best at 256-token chunks, while context precision fell steadily from 0.428 to 0.172 as chunks grew.
Abstract
Retrieval‑augmented generation (RAG) links a large language model to a set of texts. But RAG works well only when two settings are right: how big each piece of text (a "chunk") is, and how many pieces the model reads. People often choose these by habit, not by data. In this study we tested how both settings change the results of a simple RAG pipeline. We used a closed-domain set of 120 academic-regulation documents (246,800 tokens in total). We cut the documents into chunks of four sizes (128, 256, 512, and 1024 tokens) with 25% overlap, and we also tried one version with no overlap. Each chunk was turned into a vector with all-MiniLM-L6-v2 and stored in FAISS with exact inner-product search. Llama-3.1-8B-Instruct then answered questions using the retrieved chunks, at retrieval depths k = 1, 3, 5, and 10. We used 200 question-answer pairs: 120 answerable items as a development set (used to choose all settings) and a held-out test set of 60 answerable items plus 20 unanswerable items (used only once). On the development set, retrieval recall was best at 256-token chunks (recall = 0.825 at k = 5), while context precision fell steadily from 0.428 to 0.172 as chunks grew. Answer F1 was highest at k = 5 (71.3) and then dropped to 69.2 at k = 10, even though recall kept rising. Our final configuration was L = 256, O = 64 (25% overlap), k = 5. On the held-out test set this configuration reached an F1 of 70.1. That is 37.5 points higher than a closed-book baseline (p < 0.001) and 8.9 points higher than a BM25 baseline (p = 0.004). Correct abstention on unanswerable questions improved from 15.0% to 70.0%. Manual error analysis showed that 17.9% of the failures were caused by chunk-boundary fragmentation.
Web retrieval-augmented generation (RAG) splits a page into fixed-size text chunks or, in recent visual pipelines, screenshot tiles, and the retrieved item serves at once as the unit of indexing, ranking, and reading. We trace answer evidence from source HTML to reader input over the full English Wikipedia and find tha...
Peichun Hua, He Sun, Zirui Zhuang et al.· 0 citations
It is shown that gains measured in isolation are not reliable predictors of realistic performance, and that new RAG techniques should be tested with the full surrounding stack, and called on researchers to do so and supply LASAGNA to simplify these experiments.
It is found that no single retriever family dominates across documents; parser and chunker choice interact significantly; MPNet-base is a consistent underperformer with a severe failure mode on table-derived questions; and the corpus exhibits a near-saturated evidence-preservation ceiling above 98%, indicating that ret...
Portable Document Format (PDF) files are the most common way of storing reports, research papers, manuals, and legal documents, but their layout-oriented structure makes them difficult to search and to query conversationally. Large Language Models (LLMs) can answer questions fluently, but they may hallucinate and canno...
Prof. Dr. Aniruddha Rumale, Dr. Sushma Zade, Lokesh Patil, Prashik Bhise, Hareshwar Wagh, Sarthak Shende· International Journal of Adv...· 0 citations
Retrieval-Augmented Generation (RAG) effectively separates knowledge retrieval from generation for private document question answering. This study explores the design space of document-grounded QA and details a multi-PDF RAG system. The pipeline uses PyMuPDF for text extraction with OCR fallback, BAAI/bge-small-en-v1.5...
P. S. Togrikar, Tejas Pawar, Akshay Malgunde et al.· International Journal of Inn...· 0 citations
In standard question scenarios, the closest sentence is not always the correct one, so retrieval units centered on structural clauses are constructed, to address this issue.
Liang-Xu Hao, Xiao-Jing Liu, Kai-Xin Zhang· The International Conference...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.