Skip to content
Open access

Chunk Size and Retrieval Depth Optimization in a Minimal Retrieval-Augmented Generation Pipeline

Sep 2026 · International Journal of Innovative Science & Technology · 0 citations · 13 references

TL;DR

This study tested how both settings change the results of a simple RAG pipeline, finding retrieval recall was best at 256-token chunks, while context precision fell steadily from 0.428 to 0.172 as chunks grew.

Abstract

Retrieval‑augmented generation (RAG) links a large language model to a set of texts. But RAG works well only when two settings are right: how big each piece of text (a "chunk") is, and how many pieces the model reads. People often choose these by habit, not by data. In this study we tested how both settings change the results of a simple RAG pipeline. We used a closed-domain set of 120 academic-regulation documents (246,800 tokens in total). We cut the documents into chunks of four sizes (128, 256, 512, and 1024 tokens) with 25% overlap, and we also tried one version with no overlap. Each chunk was turned into a vector with all-MiniLM-L6-v2 and stored in FAISS with exact inner-product search. Llama-3.1-8B-Instruct then answered questions using the retrieved chunks, at retrieval depths k = 1, 3, 5, and 10. We used 200 question-answer pairs: 120 answerable items as a development set (used to choose all settings) and a held-out test set of 60 answerable items plus 20 unanswerable items (used only once). On the development set, retrieval recall was best at 256-token chunks (recall = 0.825 at k = 5), while context precision fell steadily from 0.428 to 0.172 as chunks grew. Answer F1 was highest at k = 5 (71.3) and then dropped to 69.2 at k = 10, even though recall kept rising. Our final configuration was L = 256, O = 64 (25% overlap), k = 5. On the held-out test set this configuration reached an F1 of 70.1. That is 37.5 points higher than a closed-book baseline (p < 0.001) and 8.9 points higher than a BM25 baseline (p = 0.004). Correct abstention on unanswerable questions improved from 15.0% to 70.0%. Manual error analysis showed that 17.9% of the failures were caused by chunk-boundary fragmentation.

Read PDF

Similar papers

Preprint Oct 2026

Tessera: Structure-Completed Evidence Units for Web Retrieval-Augmented Generation

Web retrieval-augmented generation (RAG) splits a page into fixed-size text chunks or, in recent visual pipelines, screenshot tiles, and the retrieved item serves at once as the unit of indexing, ranking, and reading. We trace answer evidence from source HTML to reader input over the full English Wikipedia and find tha...

Peichun Hua, He Sun, Zirui Zhuang et al. · 0 citations
Preprint Oct 2026

Retrieval Augmented Generation Compositionality

It is shown that gains measured in isolation are not reliable predictors of realistic performance, and that new RAG techniques should be tested with the full surrounding stack, and called on researchers to do so and supply LASAGNA to simplify these experiments.

Ido Levy, Segev Shlomov, Leshem Choshen · 0 citations
#artificial intelligence Review Sep 2026

Parser, Chunking, and Embedding Interactions in Retrieval-Augmented Generation over Indian Government Regulatory Documents

It is found that no single retriever family dominates across documents; parser and chunker choice interact significantly; MPNet-base is a consistent underperformer with a severe failure mode on table-derived questions; and the corpus exhibits a near-saturated evidence-preservation ceiling above 98%, indicating that ret...

Shubham Kumar Singh · 0 citations
Open access Oct 2026

AI-Based PDF Chat and Summarization Engine Using Retrieval-Augmented Generation

Portable Document Format (PDF) files are the most common way of storing reports, research papers, manuals, and legal documents, but their layout-oriented structure makes them difficult to search and to query conversationally. Large Language Models (LLMs) can answer questions fluently, but they may hallucinate and canno...

Prof. Dr. Aniruddha Rumale, Dr. Sushma Zade, Lokesh Patil, Prashik Bhise, Hareshwar Wagh, Sarthak Shende · 0 citations
Review Open access Oct 2026

A Survey on Retrieval-Augmented Generation for Multi-PDF Question Answering Systems

Retrieval-Augmented Generation (RAG) effectively separates knowledge retrieval from generation for private document question answering. This study explores the design space of document-grounded QA and details a multi-PDF RAG system. The pipeline uses PyMuPDF for text extraction with OCR fallback, BAAI/bge-small-en-v1.5...

P. S. Togrikar, Tejas Pawar, Akshay Malgunde et al. · 0 citations
Conference Open access Oct 2026

Structure-aware retrieval-augmented generation for industrial standard documents

In standard question scenarios, the closest sentence is not always the correct one, so retrieval units centered on structural clauses are constructed, to address this issue.

Liang-Xu Hao, Xiao-Jing Liu, Kai-Xin Zhang · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.