Impact of Artificial Intelligence Chatbots on Student Confidence and Perceptions in a Communication Course.
TL;DR
ChatGPT 4o text-based chatbot patients improved student perceived confidence in verbal counseling skills and was valued as a convenient, low-pressure, and accessible training tool.
Abstract
Objectives
To 1) improve pharmacy students' perceived confidence in communication skills and 2) identify perceptions of counseling a text-based AI-simulated patient using ChatGPT 4o.
Methods
This study involved first-year pharmacy students in a required communications course at the University of Georgia. Four AI-simulated patients with unique characteristics and medication-related problems were developed in ChatGPT 4o. After didactic instruction, students completed a pre-survey, practiced counseling with the text-based AI patients to prepare for two separate graded standardized patient cases (GSPCs) and completed identical post-surveys after each GSPC to assess changes in confidence and overall perceptions. Quantitative data was analyzed using the Wilcoxon Signed Rank Test and thematic analysis was used for open-ended responses.
Results
Of 146 students, 117 (80.1%) completed all three surveys with significant improvements in perceived confidence observed across all 7 communication domains. The greatest gains were noted in initiating counseling sessions, applying prime questions, and using probing techniques in the first post-survey. While significant in the first post-survey, no significant changes were noted in nonverbal communication skills (body language, active listening) in the second post-survey. Thematic analysis of 386 comments identified five benefit themes (e.g., realistic practice, accessibility, structured counseling), four limitation themes (e.g., lack of human-like interaction, response limits), and four improvement areas (e.g., expanded scenarios, enhanced feedback).
Conclusion
ChatGPT 4o text-based chatbot patients improved student perceived confidence in verbal counseling skills and was valued as a convenient, low-pressure, and accessible training tool. While limitations in realism and nonverbal skill development were noted, text-based AI-simulated patients represent a scalable supplement to traditional communication training in pharmacy education.