Performance and concordance of contemporary large language models on Royal Australasian College of Physicians examination questions: an exploratory analysis.
We assessed six publicly available large language models (LLM) on Royal Australasian College of Physicians (RACP) adult medicine written examination questions. In a 30-question set with answers, models achieved 93% correct responses overall, and accuracy was higher for text-based than visual-media questions. In a 170-q...