A Hybrid Approach for Personal Trainer Chatbots: Fine-Tuning and Retrieval-Augmented Generation with GPT-4o

Abstract

Despite growing awareness of the importance of a healthy lifestyle, many people still find it difficult to exercise regularly due to lack of knowledge and resources. Although personal trainers can address this issue, their services are not affordable for everyone. Large Language Models (LLMs) offer a way to address this issue by providing accessible, on-demand fitness guidance. However, LLMs still suffer from hallucination. In the context of fitness coaching, they must also deliver natural and supportive communication to maintain user engagement and motivation. To address the issues, this study combines Supervised Fine-Tuning (SFT) with Retrieval-Augmented Generation (RAG) using PaperQA to create a Gym Personal Trainer Chatbot based on LLMs, specifically GPT-4o. The system is assessed using RAGAS metrics and user satisfaction surveys, achieving Context Recall of 0.950, Faithfulness of 0.909, Answer Relevance of 0.842, and Context Relevance of 0.950, along with an average user satisfaction score of 4.42 out of 5. The findings indicate that RAG increases factual accuracy and decreases hallucinations, while SFT improves natural and comfortable interaction.