T. Haferlach

The benchmark illusion: on the curious underestimation of AI and the uncertainty of its creators’ gold standards

Summary Artificial intelligence (AI) systems have quietly crossed an uncomfortable threshold: on a growing range of diagnostic tasks, they often match, and in some well-defined tasks even exceed, human performance. Evidence from large language models such as Med-PaLM and GPT-4, and deep learning systems in pathology, r...