RAG-Based Automated Question Generation System for the Learning Management System
Abstract
Automatic Question Generation (AQG) can support learning assessment by automatically generating questions from instructional materials. However, Large Language Model (LLM) based AQG systems often produce outputs that are not fully aligned with source documents. Retrieval Augmented Generation (RAG) addresses this limitation by incorporating relevant contextual information prior to the generation process. This study develops a RAG based AQG system for the Learning Management System using locally deployed language models through Ollama. To improve retrieval quality, PDF learning modules are first preprocessed using Docling to produce structured textual content before document chunking and indexing. The system employs the nomic-embed-text embedding model and evaluates three document chunking strategies (Fixed, Recursive, and Semantic Chunking) together with two open source language models, Gemma 3:4B and Gemma 4:E4B. Performance is evaluated using the RAGAs framework based on Faithfulness, Context Precision, Context Recall, and Answer Relevancy, while representative generated multiple choice questions are qualitatively reviewed to assess their educational suitability. Experimental results show that the best performing configuration, combining Fixed Chunking and Gemma 3:4B, achieved the highest overall RAGAs score of 0.7196. The findings indicate that document preprocessing, retrieval configuration, and language model selection substantially influence retrieval-grounded question generation. Furthermore, the study demonstrates the feasibility of integrating locally deployed RAG-based AQG into Moodle-based Learning Management Systems, supporting secure, cost-effective, and trustworthy educational AI for government-operated learning environments.