Performance of large language models in pediatric infectious disease education: a multi-condition comparative study.

Background

Evaluating how appropriate current large language models (LLMs) respond to common pediatric infectious disease (PID) questions is important for understanding their potential role in education and clinical support. Therefore, we evaluated and compared the clinical quality of answers produced by different LLMs when given professionally written, clinician-style case descriptions, using a purpose-built multi-domain evaluation tool.

Methods

Three different LLMs, including DeepSeek-V3.2-Exp, ChatGPT-5, and Gemini 2.5 Flash, were investigated using 12 relevant PID scenarios. Each scenario included five open-ended questions: etiology, clinical features, diagnostics, treatment options, and complications. Two PID specialists independently reviewed all responses using a Likert scoring tool.

Results

All three models received high overall scores. ChatGPT-5 achieved perfect domain scores (10.0), whereas DeepSeek-V3.2-Exp showed similarly strong results, with only minor reductions in diagnostics (9.83) and treatment (9.92). Gemini 2.5 Flash demonstrated higher variability, particularly in treatment (8.83) and diagnostics (9.75), while matching the other models in clinical findings and complications (10.0) and scoring 9.83 in etiology. However, there were no statistically significant differences across models in domain-level comparisons (all p > 0.05). In addition, model-level analyses similarly demonstrated high accuracy (6.0; 5.98; 5.85), completeness (3.0; 2.97; 2.87), and safety (1.0; 1.0; 0.97) for ChatGPT-5, DeepSeek-V3.2-Exp, and Gemini 2.5 Flash, respectively (accuracy: p = 0.074; completeness: p = 0.066; safety: p = 0.14).

Conclusion

Current LLMs performed highly in structured, expert-reviewed PID scenarios and may assist with information organization and early-stage clinical reasoning. However, their optimal use remains within clinician-supervised workflows, where AI-generated outputs complement, rather than replace, human clinical judgment. IMPACT Current large language models (LLMs) show consistently high-quality performance across pediatric infectious disease scenarios, with no significant differences between models. Our study adds a structured, multi-domain comparison using clinician-style cases and expert evaluation, providing detailed insights into accuracy, completeness, and safety. The findings support the use of LLMs in medical education and early clinical reasoning, while highlighting their role as clinician-supervised tools rather than replacements for clinical judgment. The results demonstrate the reliability of LLMs in organizing clinical information and identifying key diagnostic and management elements in standardized scenarios.