An Optimization Framework for Retrieval Augmented Generation in Indonesian Educational Question Answering

Evaluating RAG approaches for Question Answering

Context-Aware Large Language Model for Customer Support Chatbots

Despite their high conversational fluency, large language models (LLMs) tend to produce responses that are either contrived or factually inaccurate, a phenomenon known as hallucination. This constrains their dependability in areas where accuracy is crucial, like customer service. This study leverages a context-aware chatbot built on a Retrieval-Augmented Generation (RAG) pipeline to solve the problem. The system retrieves semantically relevant text from an external knowledge base by integrating neural information retrieval with LLMs to ensure factual correctness and enhanced contextual relevance. These externally retrieved documents are given as a reference during response generation. The LLM-as-a-judge approach is used to evaluate the system by comparing responses to a qualitative performance matrix using GPT-4o. Results show that the RAG-based chatbot increases context precision by a factor of 7.5, decreases hallucinations (measured through Faithfulness) by 73.20% and improves answer relevance by 6.97% when compared to a random retrieval baseline system. This study advances conversational AI by demonstrating how the retrieval method significantly enhances the usefulness and reliability of LLMs for enterprise-level customer service. The results show that the RAG architecture provides a scalable alternative for creating precise, contextually grounded conversational agents, thereby mitigating some of the main drawbacks of LLMs.

Optimizing Context Injection for Educational Chatbots through Comparative Sentence and Recursive Chunking Strategies

This research addresses the common challenge of a lack of context in Question and Answer (QA) datasets in digital education, which limits the reasoning potential of Large Language Models (LLMs). To address this, we optimize an automated retrieval-based dataset generation system that systematically enriches QA pairs with relevant pedagogical context from authoritative digital textbooks. This study conducts a comparative analysis of two major text chunking strategies: sentence chunking and recursive chunking. Although these pipelines are designed for general education applications, they are evaluated here through a case study of Indonesian elementary education materials. To ensure the highest reliability, the workflow performance is measured against a ground truth dataset of 978 entries, manually curated and validated by education experts to ensure pedagogical accuracy, and 781 entries from other subjects. Quantitative evaluation using BERTScore shows that recursive chunking achieves a superior F1 score of 0.748 compared to 0.737 for sentence chunking, with peak performance observed on upper elementary school materials (Grades 5 and 6). These findings were corroborated by the final verification phase through User Acceptance Testing (UAT) with an elementary school educator, where recursive chunking achieved a 'Relevant' score of 22 compared to 17 for sentence chunking. A key contribution of this study is the development and validation of a standardized, automated workflow by experts that effectively overcomes the barriers of manual dataset construction for domain-specific tasks, providing a semantically robust foundation for context-aware educational AI.

Retrieval-Augmented Generation Architecture for Indonesian Academic Regulation Question Answering: A Microservices-Based Implementation

Indonesian higher-education institutions operate under a dense and frequently updated body of academic regulations—national standards, institutional statutes, and study-program handbooks—that students and staff must consult accurately. General-purpose large language models answer such questions fluently but without provenance, and they hallucinate rules that do not exist. This paper proposes a retrieval-augmented generation (RAG) architecture for Indonesian academic-regulation question answering, deployed as a set of loosely coupled microservices: an API gateway, a RAG orchestrator, an embedding and retrieval subsystem backed by a FAISS vector index, a generation subsystem, and an offline ingestion pipeline. The retrieval layer combines lexical BM25 and vector similarity through reciprocal rank fusion, so that regulation passages are grounded and citable. We evaluate the retrieval core—the component that determines whether generated answers can be grounded—on an original corpus of 20 synthetic Indonesian academic-regulation passages and 30 labeled questions. Vector retrieval attains Recall@1 of 0.867, MRR@10 of 0.902, and nDCG@5 of 0.915, outperforming BM25 (0.800, 0.865, 0.898) on early precision, while BM25 is an order of magnitude faster and reaches perfect Recall@5. Retrieval latency for all configurations remains below one millisecond on a single node. The microservices decomposition lets the index be rebuilt when regulations change without redeploying the generation service. Results indicate that a hybrid retrieval core is a sound and inexpensive foundation for grounded, citable regulation question answering, and that the architecture is deployable on modest institutional infrastructure.

An Intelligent PDF Question-Answering System; A Retrieval-Augmented Generation Approach

Due to the surge in digital document creation, intelligent systems that can effectively retrieve accurate information from unstructured PDF content have become a necessity. Current keyword-based search techniques lack semantic meaning, while large language models (LLMs), when utilized individually, produce hallucinated results when they are not contextually aware of document content. In this paper, we introduce a novel real-time question answering system for PDF content using a lightweight approach to Retrieval-Augmented Generation (RAG). Our system combines a HuggingFace sentence transformer model for semantic embeddings with a FAISS vector similarity search for efficient retrieval, leveraging a Groq-hosted LLaMA 3.3 70B model for context-grounded answer generation. The system is built using LangChain for orchestration and Streamlit for web-based deployment, and requires no GPU, no fine-tuning, and no pre-indexed corpus. Evaluation across five complementary metrics—Exact Match, Token-level F1, BLEU, ROUGE, and semantic similarity—demonstrates strong alignment between generated and reference answers, with a peak semantic similarity of 0.8709 and a mean Token F1 of 0.4701, confirming the viability of this approach for real-world document intelligence.

Rhetorical-Role-Aware Retrieval-Augmented Generation for Legal Question Answering over Indian Supreme Court Judgments

This research paper proposes a Retrieval Augmented Generation framework that is specific to the legal field in order to assist interactive retrieval and reason about judgments from the Supreme Court of India and demonstrates strong performance on metrics including contextual recall and answer relevancy.