Development of an Information Service Chatbot for University Websites Based on Natural Language Processing (NLP) and Retrieval-Augmented Generation (RAG)
It is concluded that an NLP- and RAG-based chatbot is feasible to be implemented as a digital campus information service and improves answer accuracy from 58.5% to 89.25%, with an average response time below three seconds.
Abstract
Academic information services on university websites still largely rely on static systems and complex navigation menus, making it difficult for students and prospective students to obtain information quickly. This study aims to develop an information service chatbot for the Universitas Serambi Mekkah website that integrates Natural Language Processing (NLP) and Retrieval-Augmented Generation (RAG) in an end-to-end manner. The Research and Development (R&D) method was applied, covering system design, implementation, and testing. The system was built using a client-server architecture with React.js on the frontend and FastAPI on the backend, the text-embedding-3-small model from OpenAI for vector representation, ChromaDB as the vector database, and a Large Language Model accessed through OpenRouter for answer generation. Official university documents were extracted using PyPDF as the knowledge base. The testing results indicate that integrating NLP and RAG improves answer accuracy from 58.5% (baseline LLM without RAG) to 89.25%, with an average response time below three seconds. User satisfaction testing obtained an average score of 4.22 out of 5 (very good category). This study concludes that an NLP- and RAG-based chatbot is feasible to be implemented as a digital campus information service. Note: the quantitative results reported here are illustrative and must be replaced with actual field measurements prior to final publication.
Academic libraries increasingly require intelligent digital services to improve information accessibility, responsiveness, and service efficiency. However, conventional FAQ-based chatbots often suffer from limited semantic understanding, while fully AI-based systems may introduce higher latency and inconsistent institutional responses. This study aims to design and evaluate a hybrid web-based library chatbot integrating a local MySQL FAQ database with OpenAI API services to support digital library services at Universitas Negeri Padang (UNP). The system was developed using a hybrid architecture combining semantic FAQ retrieval and AI-based fallback generation. The implementation utilized HTML, CSS, JavaScript, Node.js, and MySQL, while Bahasa Indonesia Natural Language Processing (NLP) techniques, including normalization, tokenization, stopword removal, and stemming, were applied to improve semantic query matching. Evaluation was conducted using 100 library-related queries, validated by three experts: senior librarians and an information systems lecturer. Experimental results demonstrated that the proposed hybrid chatbot achieved 92% response correctness accuracy with a 95% confidence interval ranging from 86.7% to 97.3%. Routing evaluation also confirmed effective query classification between local FAQ retrieval and AI fallback mechanisms, while benchmarking experiments showed that the proposed architecture outperformed keyword-based FAQ systems, semantic retrieval-only models, and pure LLM-based chatbots in terms of response accuracy and balanced response latency. The findings indicate that integrating semantic retrieval with AI-based fallback mechanisms can improve response quality, scalability, and contextual relevance in academic library services. Future research may focus on adaptive threshold optimization, multilingual support, and domain-specific prompt engineering to further enhance routing precision and the generation of contextual responses.
Asrul Huda, Ella Dwipujana Asyani, Firdaus Firdaus et al.· JOIV: International Journal...· 0 citations
Universities now use Large Language Models (LLMs) to transform their processes for managing student information. The paper introduces an upgraded chatbot system for Narasaraopeta Engineering College (NEC) which extends previous on-premise LLM chatbot research by providing four new functions. The system uses (1) Retrieval-Augmented Generation (RAG) to create citation-based responses through LlamaIndex and ChromaDB, (2) Context Memory which maintains conversation flow during multiple dialogue exchanges, (3) Voice Input through OpenAI Whisper Speech-to-Text (STT) technology, and (4) Multilingual Support which covers English and these seven languages: Hindi, Telugu, Tamil, Kannada, and Malayalam through IndicNLP. The system tested 60 benchmark questions across four academic categories which included regulations and examination policies and fee structures and multilingual queries and achieved 96.7% overall accuracy with sub-second text response times and 1.0–1.4 second voice response times. The system operates entirely on-premise through Docker which safeguards institutional data privacy while eliminating the need for recurring cloud API expenses. The upcoming development will create Emotion-Aware AI, FAQ Auto-Learning, Student Portal Integration, and a Mobile Application.
M. Yaswanth, Kopparapu Sai Amar Durgesh, Mogili Harsha Vardhan et al.· 2026 7th International Confe...· 0 citations
Academic information services in higher education institutions still face challenges because important information is often distributed through static, lengthy, and difficult-to-navigate PDF regulatory documents. This condition makes it difficult for students to obtain specific academic information efficiently and may increase repetitive inquiries directed to academic administrative staff. This study aims to develop and evaluate a Retrieval-Augmented Generation (RAG)-based chatbot for academic regulation information services using a locally deployed Large Language Model (LLM). The main contribution of this study lies in the implementation of a local RAG architecture for hierarchical academic regulation documents by combining hybrid chunking, hybrid retrieval, reranking, and evaluation using retrieval metrics and RAGAS. The proposed method includes document preprocessing, document segmentation with variations in chunk size and chunk overlap, embedding generation, storage in ChromaDB, hybrid retrieval combining semantic and lexical search, and integration of Gemma 3:4B through Ollama. System evaluation was conducted through six experimental scenarios with different retrieval configurations. Retrieval performance was measured using Precision, Recall, and Mean Reciprocal Rank (MRR), while response quality was evaluated using Faithfulness, Answer Relevancy, Context Precision, and Context Recall. The best configuration was achieved with Top-K 10, chunk size 1024, and chunk overlap 400, producing scores of 0.7950, 0.7868, 0.8093, and 0.8958, respectively. These results indicate promising feasibility for supporting document-based academic regulation information services, although broader validation is still required due to the limited evaluation dataset and restricted document scope.
Academic information services in higher education institutions still face various obstacles, such as delays in information delivery, limited access to services, and high administrative burdens due to repetitive student inquiries. This study aims to implement Retrieval-Augmented Generation (RAG) and Semantic Search technology in a web-based Artificial Intelligence chatbot to optimize academic services at the Faculty of Engineering, Hamzanwadi University. The research method used is Design Science Research (DSR), which includes data collection, system requirements analysis, design, implementation, testing, and system evaluation. The chatbot's knowledge base is built from academic documents such as academic guidelines, service SOPs, academic calendars, scholarship information, and other administrative documents. The system was developed using an integration of LangChain, Azure OpenAI Service, Azure AI Search, FastAPI, Next.js, and Supabase. Semantic Search techniques are used to perform vector embedding-based searches, while RAG is utilized to generate contextual answers based on relevant documents. Test results show that all key system features performed well with a 100% success rate in unit testing. A user satisfaction evaluation using the Customer Satisfaction Index (CSI) method with 54 respondents yielded a score of 89.34%, categorized as "Very Satisfied," with a Mean Satisfaction Score above 4.37 on a maximum scale of 5.0. The AI chatbot successfully addressed traditional academic information service issues by providing 24/7 service, reducing the workload of campus staff, and ensuring information consistency through RAG technology.
Muhammad Saiful, L. Samsu, Imam Fathurrahman et al.· Infotek : Jurnal Informatika...· 0 citations
Results suggest that code-mixing-aware RAG chatbots can meaningfully improve DSA learning outcomes while remaining feasible for instructor-led deployment in low-resource settings.
Saroj Bhandari, Puja Dhital· Aadim Journal of Multidiscip...· 0 citations
This paper presents the design and implementation of intelligent chatbots using Retrieval-Augmented Generation architecture to derive accurate, context-based responses from domain-specific knowledge sources such as PDFs and CSV files. The proposed system integrates information retrieval mechanisms with large language models to overcome crucial limitations of traditional chatbots: hallucination, knowledge cut-off, and inability to access private organizational data. The proposed architecture uses a six-stage pipeline consisting of document ingestion, text splitting, and embedding generation with sentence transformers, semantic retrieval, and response generation via Large Language Models (LLMs). By preprocessing the PDF and CSV documents using a RecursiveCharacterTextSplitter and converting them into 384-dimensional semantic embeddings, the system works via a similarity-based retrieval of document chunks. For implementation, the LlamaIndex framework was used for data indexing and query orchestration, and HuggingFace Transformers for language modeling. Experimental validation conducted on the smart campus chatbot system demonstrated significant improvements in the accuracy, relevance and transparency of responses across education, student services, and administrative support domains compared with a standalone language model. Because of its modular design, it supports a wide range of domain-specific topics without the combinatorial costs of re-training. This is another important application for systems that require regular updates and accurate information. Overall, the proposed framework provides a simple and practical approach for easily implementing domain-aware chatbots.
Varsha Spoorthi Shree Narravula, V. K, Yashaswini B. Suresh et al.· 2026 7th International Confe...· 0 citations