Abstract Clinical artificial intelligence (AI) has advanced rapidly, with frontier large language models now matching or exceeding physician performance on simulated diagnostic reasoning and clinical decision-support tasks. Yet adoption has outpaced the evidence base: fewer than 5% of cleared U.S. Food and Drug Adminis...
J. Worth, Anastasia Perez, David Wu et al.· BMJ digital health & AI· 1 citation
This article aims to provide a pragmatic basis for designing and executing rigorous evaluations of healthcare LLMs by describing underlying concepts and potential pitfalls, while emphasizing the importance of aligning evaluation methods with the research question.
S. E. McKinney, P. Vu, S. Justice et al.· 0 citations
Blinded physician evaluation has been considered by many to be the gold standard for assessing clinical reasoning in large language models (LLMs). This is difficult to scale; thus, prior studies typically rely on small physician panels, often from a single institution or specialty, which both limits the scientific ques...
Thomas A. Buckley, Zahir Kanjee, Peter G. Brodeur et al.· 0 citations
On HealMed, performance declined most in low-resource languages, although the size of the gap varied markedly across languages and models, whereas many open-source and medically specialized models showed larger and less consistent gaps.
Yingjian Chen, Fan Gao, Sherry T. Tong et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
