Optimizing Context Injection for Educational Chatbots through Comparative Sentence and Recursive Chunking Strategies

Abstract

This research addresses the common challenge of a lack of context in Question and Answer (QA) datasets in digital education, which limits the reasoning potential of Large Language Models (LLMs). To address this, we optimize an automated retrieval-based dataset generation system that systematically enriches QA pairs with relevant pedagogical context from authoritative digital textbooks. This study conducts a comparative analysis of two major text chunking strategies: sentence chunking and recursive chunking. Although these pipelines are designed for general education applications, they are evaluated here through a case study of Indonesian elementary education materials. To ensure the highest reliability, the workflow performance is measured against a ground truth dataset of 978 entries, manually curated and validated by education experts to ensure pedagogical accuracy, and 781 entries from other subjects. Quantitative evaluation using BERTScore shows that recursive chunking achieves a superior F1 score of 0.748 compared to 0.737 for sentence chunking, with peak performance observed on upper elementary school materials (Grades 5 and 6). These findings were corroborated by the final verification phase through User Acceptance Testing (UAT) with an elementary school educator, where recursive chunking achieved a 'Relevant' score of 22 compared to 17 for sentence chunking. A key contribution of this study is the development and validation of a standardized, automated workflow by experts that effectively overcomes the barriers of manual dataset construction for domain-specific tasks, providing a semantically robust foundation for context-aware educational AI.