Comparison of Chatbot Responses to Frequently Asked Patient Questions in Periodontology

Abstract

Aim: This study compared the accuracy, quality, understandability, actionability, and readability of responses generated by ChatGPT-4o, Gemini-2.5 Flash, and DeepSeek-V3 to common patient questions related to periodontal diseases. Material and Methods: Twenty open-ended patient-based questions were submitted to each chatbot. Two periodontologists independently evaluated the responses for scientific accuracy, overall quality (GQS), understandability, and actionability (PEMAT-P), while readability was calculated using the Ateşman Readability Index. Inter-rater reliability was good to excellent for the expert-rated domains. Results: No significant differences were found among chatbots in accuracy or overall quality (p > 0.05). Significant differences were observed in understandability, actionability, and readability (p < 0.001). ChatGPT-4o and DeepSeek-V3 showed comparable understandability and outper-formed Gemini-2.5 Flash, whereas ChatGPT-4o and Gemini-2.5 Flash showed comparable ac-tionability and outperformed DeepSeek-V3. For readability, ChatGPT-4o achieved the highest scores, followed by DeepSeek-V3 and Gemini-2.5 Flash. Conclusion: Although all chatbots produced accurate responses, their performance varied across communication domains. ChatGPT-4o showed the most favorable overall profile, particularly in readability and patient-oriented clarity; however, no single model should be considered uniform-ly superior across all criteria.