Context-Aware Large Language Model for Customer Support Chatbots

Abstract

Despite their high conversational fluency, large language models (LLMs) tend to produce responses that are either contrived or factually inaccurate, a phenomenon known as hallucination. This constrains their dependability in areas where accuracy is crucial, like customer service. This study leverages a context-aware chatbot built on a Retrieval-Augmented Generation (RAG) pipeline to solve the problem. The system retrieves semantically relevant text from an external knowledge base by integrating neural information retrieval with LLMs to ensure factual correctness and enhanced contextual relevance. These externally retrieved documents are given as a reference during response generation. The LLM-as-a-judge approach is used to evaluate the system by comparing responses to a qualitative performance matrix using GPT-4o. Results show that the RAG-based chatbot increases context precision by a factor of 7.5, decreases hallucinations (measured through Faithfulness) by 73.20% and improves answer relevance by 6.97% when compared to a random retrieval baseline system. This study advances conversational AI by demonstrating how the retrieval method significantly enhances the usefulness and reliability of LLMs for enterprise-level customer service. The results show that the RAG architecture provides a scalable alternative for creating precise, contextually grounded conversational agents, thereby mitigating some of the main drawbacks of LLMs.