The Impact of Artificial Intelligence-Based Training: A Pre-Post Evaluation of Knowledge and Satisfaction Among Master's Medical Students

Abstract

AI-based technologies are rapidly being adopted in healthcare. This underscores the need for AI literacy among physicians and healthcare professionals, encompassing core knowledge, practical skills, and ethical awareness. This study assessed a 2-credit course, “Artificial Intelligence in Healthcare,” given to graduate-level medical students. A one-group pretest-posttest quasi-experimental design was used. Thirty-nine students completed both the knowledge assessment and the satisfaction survey. Students' knowledge scores rose from pretest $(M=7.13)$ to posttest $(M=8.10)$, with a mean difference of 0.97 (95% CI: $0.62-1.31; \mathrm{t}(38)=5.72,\ \mathrm{p}<0.001)$, showing a large effect size (Cohen's $\mathrm{d}=0.92)$. Student satisfaction was high (Mean = 4.63-4.81; Cronbach's $\alpha=0.951)$, indicating the course content met learner needs. Findings suggest short-term, practice-oriented AI education boosts AI literacy. Key improvements were seen in prompt engineering, ethical awareness, and clinical application. More research is needed, especially with performance-based assessments and multi-center designs.