Are AI Chatbots Ready for HoLEP Patient Education? A Comparative Evaluation of ChatGPT, Gemini, and Claude

Abstract

Objective: To compare the quality, guideline concordance, and readability of HoLEP-related patient education responses generated by ChatGPT, Gemini, and Claude.Materials and Methods: Thirty standardized patient questions on holmium laser enucleation of the prostate (HoLEP) were submitted independently to three conversational AI platforms, generating 90 responses. Response quality was evaluated using the 16-item DISCERN instrument, and clinical accuracy was assessed with question-specific four-item guideline-concordance checklists. Readability was measured using seven validated formulas. Platform differences were analyzed using the Friedman test and Bonferroni-corrected Wilcoxon signed-rank tests.Results: Significant differences were found among platforms for DISCERN scores, checklist performance, and all readability metrics. Claude achieved the highest mean DISCERN and checklist scores, followed by ChatGPT, whereas Gemini scored significantly lower. Claude also produced the most readable responses across all formulas. However, all three platforms exceeded the recommended sixth to eighth-grade reading level for patient education materials.Conclusion: Claude and ChatGPT provided higher-quality and more guideline-concordant HoLEP-related information than Gemini, while Claude showed the best readability. Nevertheless, all platforms remained above recommended patient readability levels and produced only poor-to-moderate DISCERN scores. Conversational AI may support HoLEP patient education, but clinician review, simplification, and guideline-based verification remain necessary.